Category Archives: Technology

Python2 & Python3 on the same Apache server

Posted on by

Notes on setting up a python 3 application on an apache server already running python 2 applications. This covers Flask, mod_wsgi-express, Apache2 & HTTPS support. 

For my latest personal project – a web app allowing my wife to fetch activities tracked on her garmin GPS watch & upload them to runkeeper – I finally decided to take the plunge & switch to Python 3.(1)When I began my PhD way back in 2012 and started learning Python, library support for Python 3 wasn’t yet complete enough for me and I’ve been stuck on Python 2 since. All was well when testing locally using Flask’s built in webserver, but things got tricky when I tried to deploy on my VPS where Apache was already serving Python 2 applications.

Running both isn’t directly possible, though the linked page starts to explain the solution. I found bits of this scattered across the web, but no complete descriptions. Hopefully this post might help others in a similar position. Throughout, I will show the setup using paths on my system, with a Flask application deployed in /var/www/apps/garmin with a wsgi script called flask.wsgi, which we want to serve on the URL /garmin .

mod_wsgi-express

What makes this possible at all is mod_wsgi-express. Using the standard mod_wsgi, Apache uses an embedded Python, which prevents running two versions simultaneously. Instead, mod_wsgi-express configures its own apache instance, which can target whatever Python version we like. There is a pip module, which provides mod_wsgi-express though you will need python & apache headers installed (e.g.  apt-get install python3-dev apache2-dev).

We can then start a new web server using the following:

This of course needs to be run as a user that can access these files. The –reload-on-changes means that we do not have to restart the server after making changes to our application. A new apache server will be listening on localhost:8000 by default, with the configuration & service scripts available in a directory something like /tmp/mod_wsgi-localhost:8000:106.

Apache2 reverse proxy

With mod_wsgi-express providing a local Apache server, we now need our public-facing Apache instance to forward requests. We set up a Reverse Proxy to forward requests to the appropriate resource to the internal webserver. We can do so this like so:

Unlike standard mod_wsgi, there is no need for us to configure WSGIDaemonProcess, WSGIScriptAlias, WSGIProcessGroup, WSGIApplicationGroup etc. Great!

Fixing the URLs

But wait! Although this appears to work and our site is being served to the world, all of our links that are relative to our site root (i.e. beginning with a /) are broken and point to the public-facing site root. This includes URLs generated by Flask. Fortunately, the fix is quite straightforward – though quite a few guides seem to have this slightly wrong. First, we update the public-facing apache configuration:

We then provide the –mount-point flag to mod_wsgi-express:

Finally, we need to make sure xhr requests understand where to route to. We can do this by passing through the script root in our base Flask template:

We then simply prefix this to any site-relative links generated Javascript.

HTTPS support

The final challenge was providing HTTPS support – since we’re forced to scrape Garmin’s website, users must provide their garmin username and password. With an SSL certificate in place, at least they’re not sent in the clear(2)Yes, this really isn’t nice – originally I’d planned to keep this a private application for my wife and keep her credentials on the server, but this at least means it could be used by more than one person and means I don’t have to authenticate users myself. Using the absolutely awesome Let’s Encrypt, I setup a free, signed HTTPS certificate.

The problem is that the internal, forwarded request is a standard HTTP request (not really a security issue; all internal traffic). However, this means URLs constructed by Flask will use HTTP and not HTTPS. This might not be a problem for most people if you’re forwarding HTTP requests to HTTPS but it’s a huge issue if the URL were to be used in, say, a hash function as in the Oauth redirect_uri! To fix, make sure the public-facing Apache sets an appropriate header:

and mod_wsgi-express knows to trust such a header:

In newer versions of Flask – I’m running 0.12.2 — this is good enough and Flask will take care of the rest. It appears that older versions might not to so automatically and you may need to experiment. However, AFAICT the ProxyFix shouldn’t be necessary as our http server is correctly setting the headers for us.

Aside: certbot doesn’t understand WSGI directives

certbot, the tool provided by Let’s Encrypt to auto-configure your setup, doesn’t yet properly understand WSGI directives. In particular, the name given to each WSGIDaemonProcess must be unique. certbot attempts to duplicate these, causing an Apache config error & certbot to rollback the changes. For now, the simplest fix is to comment out such lines before using certbot & then uncomment one of each (or both, renaming the HTTPS daemon process).

   [ + ]

1. ↑ When I began my PhD way back in 2012 and started learning Python, library support for Python 3 wasn’t yet complete enough for me and I’ve been stuck on Python 2 since
2. ↑ Yes, this really isn’t nice – originally I’d planned to keep this a private application for my wife and keep her credentials on the server, but this at least means it could be used by more than one person and means I don’t have to authenticate users myself

WiFi is not always best

Posted on by

I own a Nexus 4. Aside from a few niggles with the camera and distinctly average battery life, my phone is great. Particularly as someone who uses Google services, the user experience is pleasant and if feels like everything is working for you.

Well, almost everything. I find myself constantly fighting the phone over network connections – specifically when the phone should connect to available WiFi networks. The switch to WiFi appears extremely aggressive: as soon as a known network is within reach the phone automatically connects, regardless of the strength of the connection, the availability of internet access from the network, or – and here’s the kicker – what I’m currently doing.

Often I will be walking around Bristol, chatting with friends and family on WhatsApp or Google Hangouts whilst using mobile internet. I’ll walk past a cafe I’ve visited before and my phone switches to its WiFi network, stopping that message I was about to send. I swipe down and turn off WiFi. Later on, I’ll be home checking Facebook/Reddit/Hacker News/Any Other Time Sink and be frustrated that all these images are loading so slowly – I need to see that cute kitten now damnit – until I realise WiFi is still off. So I swipe down and re-enable.

Why is this so frustrating? Well, firstly because it is so common: it probably happens multiple times a day. Secondly, it all feels very preventable:

  • My phone is aware I’m moving(1)This doesn’t even require high battery GPS; the constant change of networks is sufficient. Why jump on a new WiFi network that is likely to drop out of range?
  • Instant messaging is a very low bandwidth activity. I really don’t care about saving my data allowance. I’m with giffgaff and have a generous data plan(2)My allowance is 3GB/month. I used to be on an unlimited data plan but I don’t mind the new cap. I rarely hit 1GB and a limited data plan allows me to tether, which is occasionally useful to me
  • Surely the phone could at least test that WiFi network actually provides internet access before connecting? Many networks are unprotected but require a login after connection. This is particularly true of the large, national networks with many hotspots around a city. Inevitably, these are the biggest problem.

Although the phone provides an option to ‘avoid poor connections’, this doesn’t seem to help much(3)If anything, it just makes thigns worse. It feels like another setting to fight with: there’s been too many times when I have wanted to connect to a network with a weak signal.. Perhaps I could use one of these apps that only enables WiFi at home and and work? Maybe, but this is definitely not tackling the problem and only fixes some of the symptoms. When I visit my friends or parents, I want to use their WiFi network. When I’m actually in a cafe, I might want to use their network. And particularly at work, which for me is actually an entire university campus, WiFi may not be providing me with a good internet connection at any given time. For me, the fix is this:

Use mobile internet unless there is a real reason not to

Where reasons include:

  • The user has indicated that they much prefer WiFi networks, e.g. because they have a small data plan
  • There is a stable connection to a known network that also actually provides a better internet connection
  • The activity I want to perform is high bandwidth. This is the time for my phone to say “hey, there’s a connection that might work, but you’ll need to log in”
  • Mobile internet coverage or speeds become poor

Doing this requires the phone to take account of how I’m currently using my phone – Am I moving? What’s my network usage look like at the moment? – as well as testing a WiFi connection before switching to that. Is that really so hard?

 

   [ + ]

1. ↑ This doesn’t even require high battery GPS; the constant change of networks is sufficient
2. ↑ My allowance is 3GB/month. I used to be on an unlimited data plan but I don’t mind the new cap. I rarely hit 1GB and a limited data plan allows me to tether, which is occasionally useful to me
3. ↑ If anything, it just makes thigns worse. It feels like another setting to fight with: there’s been too many times when I have wanted to connect to a network with a weak signal.