Converting a simple website to Python Flask

Flask is a Python web framework that's steadily increasing in popularity within the webdev world. After reading so many great things about Flask, I decided to try it out myself. I personally find testing out a new framework difficult, because you must find a project complex enough to reveal the framework's quirks, but not so daunting as to take the fun out of the project. Luckily, my PHP/Wordpress powered website filled this role quite nicely - the website simply consists of static content, a contact page, and a blog. If I could not convert such a simple site over to Flask, I would immediately know that Flask and I would not make a good team.

Spoiler alert: You're reading this on a Flask powered site! Feel free to check out the source of this site on Github.

Getting Started

The first task at hand was simply getting the home, about, services, and work pages to render correctly. While the task consisted mostly of copying and pasting, I was able to immediately apply most of the concepts from the Flask quickstart guide. Specifically, I had to learn

  • How do I actually start the server?
  • Where do my templates go?
  • Where do my static assets go?
  • How do I retrieve my assets from within the templates?
  • How do I route requests?

Sure enough, getting the pure HTML documents to render was pretty simple. By pure HTML, I mean there were no includes for headers, footers, etc. Thus the logical next step was to take these pure HTML documents and DRY up the repeating areas.

Using Jinja2

UPDATE: Turns out I was just being dumb. The include statements work fine.


Jinja2, Flask's default templating engine, uses template inheritance in additon to supporting includes. To demonstrate, suppose we have a parent template layout.html

<!DOCTYPE html>
<html lang="en">
  <head><title>My Website</title> </head>
  <body>
  <div id="wrapper">
    <div id="content">
      {% block body %}{% endblock body %}
    </div><!--content-->
  </div><!--wrapper-->
  </body>
</html>

And a child template index.html that inherits the parent and fills in its blocks

{% extends "layout.html" %}
{% block body %}
  Body content goes here!
{% endblock body %}

If we were to call render_template on index.html, this is what we would get.

<!DOCTYPE html>
<html lang="en">
  <head><title>My Website</title> </head>
  <body>
  <div id="wrapper">
    <div id="content">
      Body content goes here!
    </div><!--content-->
  </div><!--wrapper-->
  </body>
</html>

The include statement works similarly to PHP's include statement in regards to importing files that render markup. Consider the footer for this site, located in templates/includes/footer.html:

<div id="footer">
  <div class="hr"><hr /></div>
  <p>
    <a href="mailto:hi@vertstudios.com">hi@vertstudios.com</a> |
    <a href="http://www.twitter.com/vertstudios">@vertstudios</a>
  </p>
</div>

And templates/layout.html:

<!DOCTYPE html>
<html lang="en">
  {% include "includes/meta_and_css.html" %}
  <body onload="prettyPrint()" id="{{g.bodyID}}">
<div id="wrapper">
  {% include "includes/nav.html" %}
  <div id="content">
    {% block body %}{% endblock body %}
  </div><!--content-->
  {% include "includes/footer.html" %}
</div><!--wrapper-->
{% include "includes/javascripts.html" %}
</body>
</html>

Being 'Brave'

Flask didn't always have an out-of-the-box solution to the problem at hand. In fact, it rarely did. Flask did, however, give me all the tools necessary to create whatever I wanted. In a way, I felt that Flask itself was telling me "Joseph, you're smart enough to implement X functionality on your own... be brave for once!"

Sometimes reinventing the wheel is a rewarding exercise.

The Contact Form

The recommended method of dealing with forms in Flask is the Flask-WTF extension. Flask-WTF has many built in validation methods, but it did not include validation for a phone number. No big deal, any form library worth its salt has means for defining custom validation functions. WTForms does in fact have a mechanism for implementing custom validators, but I did not find their examples very pleasing to the eye:

def length(min=-1, max=-1):
    message = 'Must be between %d and %d characters long.' % (min, max)

    def _length(form, field):
        l = field.data and len(field.data) or 0
        if l < min or max != -1 and l > max:
            raise ValidationError(message)

    return _length

class MyForm(Form):
    name = TextField('Name', [Required(), length(max=50)])

Consequently, I used Flask-WTF to gather data from the form fields, but I rolled my own validation module, which only took about an hour.

The actual sending of the email was easily handled with Python's built-in smtplib module and Gmail.

The Blog

Creating a blog has become the "Hello, World" of web application frameworks. While Python blogging software does exist, it seemed like it would take longer to integrate an existing blogging platform into my Flask project than just rolling my own.

For the past year or so, I've longed for a file-system based blogging platform. Writing my articles in VIM + Markdown is so much more enjoyable than writing HTML in VIM, moving the HTML over to Wordpress, previewing the HTML, making sure the formatting is right, and other annoying steps.

There were a few specific things I needed to accomplish with the blog:

  • Ability to generate an RSS feed
  • The URLs of the blog posts must be the same as before.
  • Pagination

Without the need to generate an RSS feed, the only thing I would need to do is route /blog/<post>/ to a function that attempts to render a template named /blog/the-post-url, and return a 404 if that template isn't found. The RSS requirement, however, forced me to think how I should go about storing the meta data for each post.

I settled on a fairly simple structure that, while not beautiful or clever, works good enough for me. I create a directory in /blog/posts/ named desired -post-url. Within that directory, I create two files: meta.py and body.html. meta.py just contains a few variables for metadata (date posted, excerpt, etc), and body.html has the post body.

To avoid having to parse through all the metadata files to figure out the order of the posts for the RSS feed, I just prepend the directory name to the top of a file called rss.txt. Again, not the most beautiful thing in the world but it's a system that I find easy to understand and easy to work with.

The RSS feed and blog pagination are handled in the same step. I run a script called genfeed.py which creates the rss xml file and pagination files. Using static files generated by the geenfeed.py script (which reads the rss.txt script mentioned earlier) for pagination makes sense because the pagination should only change whenever I update/create a story.

Importing my Blog Posts

To import my Wordpress posts into this new structure, all I had to do was parse Wordpress's export xml file with PyQuery and create the files with standard library functions. I didn't bother to convert the posts from Wordpress to Markdown, so all old posts you look through in the source will have normal HTML.

Structural Troubles

One of the biggest troubles I faced throughout the site development was how to import things into particular files. This StackOverflow answer describes the steps necessary to import a module in a parent/sibling directory when running a script directly. Consequently, any scripts that needed access to the app object (which holds a lot of important methods and attributes about the application instance), I had to either run python -m mywebsite.blog.genfeed, or just move genfeed.py to the same directory as mywebsite.py, which would enable me to run python genfeed.py. I chose the latter out of sheer laziness, though I'm not particularly happy I had to make such a choice. I should note that this is not a Flask specific issue, but a Python issue.

Development vs Production Environment

In development, all assets are served from the static files. In production, all assets are served from S3. The environment is set by a env-var called FLASK_ENV. I needed a way to let my templates know "Hey, you're in the production environment, you should link to S3 files instead of static files!". I decided to take advantage of Flask's g object, which is a global object that exists througout the request context. You're free to populate it with whatever you want. I created and used the property g.assets.

While this suited my needs perfectly, it caused my templates to appear slightly less pretty. Anytime I wanted to call a macro that looks for the g.assets property, I'd have to pass g to it. For example, here's the definition of the img macro, a function used to render images on the site:

{% macro img(g, file, alt="", class="") %}
<img 
  src="{{g.assets}}/images/{{file}}" 
  alt="{{alt}}" 
  class="{{class}}" 
  />
{% endmacro %}

An example call to the macro:

{{img(g, "pointer.png", "", "pointer")|safe }}

It's more verbose than what I'd really like, but it gets the job done.

Going Live with Nginx and uWSGI

Now that my project was pretty much finished, it was time to put the files on my Linode VPS and show my project to the world! I decided to use Nginx with uWSGI, simply because the pair have been the subject of many tutorials. Flask itself has documentation on uWSGI. However, I found parts of the documentation inadequate.

From the documentation:

# Given a flask application in myapp.py, use the following command:
$ uwsgi -s /tmp/uwsgi.sock --module myapp --callable app

If you want to put a project on a live server, there's no way you should have to do so much work! uWSGI has ini configuration files that I took full advantage of. Now, instead of executing a verbose command from the shell, I can simply execute

$ uwsgi uwsgi.ini

My uwsgi.ini is as follows:

[uwsgi]
project = vertstudios
daemonize = /var/log/nginx/access.log
master = true
chdir = /var/www/vertstudios.com
socket = 127.0.0.1:5000
wsgi = %(project):variable holding app instance
virtualenv = env/
pidfile = /var/run/uwsgi/%(project).pid 
touch-reload  = /var/run/uwsgi/%(project).pid  
processes = 3 
procname-prefix = %(project)

Now my uWSGI configuration is under version control with the rest of my project.

Here's an explanation for the parameters:

  • project: The module name where you call app.run().
  • daemonize: If you want to daemonize. For some reason, you have to specify a log file path or uWSGI will just shove things down stdin. true did not work for me.
  • master: Boolean indicating if you want a master process. Master processes make uwsgi management much easier.
  • chdir: Change directory. Essentially lets you keep parameters such as "project" and virtualenv relative to the path provided to chdir.
  • socket: A Unix socket path or TCP socket info
  • wsgi: %(the module containing app):app
  • virtualenv: Path to the virtualenv for the project.
  • pidfile: Where the master process pid file will be written.
  • touch-reload: Touching this file causes the processes to reload (convenient!)
  • processes: How many processes you want running (not including master process)
  • procname-prefix: Process name prefix. Useful for ps aux | grep myapp

Now, for the parts of my nginx config relevant to uWSGI:

# Redirect everything to the main site.
server {
  server_name www.vertstudios.com;
  return 301 http://vertstudios.com$request_uri;
}

server {
  server_name vertstudios.com;
  location / { try_files $uri @flaskapp; }
  location @flaskapp {
    include uwsgi_params;
    uwsgi_pass 127.0.0.1:5000;
  }
}

Things left TODO

I felt I had done enough to merit scrapping my PHP/Wordpress site and put the Flask site up in its place. Despite that, there's still quite a bit of work to be done.

  • I plan on exporting my Wordpress comments to Disqus. I enjoy interacting with people that read my articles, and I'd like to preserve the comments I've already acquired.

  • I've managed to break a few links. In particular, I've broken links to php files I stupidly used to demo some of my jQuery plugins.

  • Using prettify for syntax highlighting means I have to still use <pre> tags in my documents, since the script requires a class of "prettyprint" on those tags. I'll either adjust prettyprint js and styles myself, find an alternative syntax highlighter, or do some processing of the markup to append the "prettyprint" class whenever it finds the <pre> tag.

  • Still need to set up git deployment. I'm currently just pulling from a private bitbucket repo on my server.

  • I need to automate my build workflow. A single bash script should concat all my CSS/JS files, run YUICompressor, and upload the files to S3.

  • I really need to implement meta description tags for my posts...but that would require me to actually write them.

Overall Impressions

As a whole, I'm quite pleased with Flask. For the most part, it gets out of your way and lets you build things how you want to build them. Consequently, your project structure may be sloppy (especially when first learning Flask), but you're guaranteed to understand how and why everything works.

July 09, 2012
About the Author:

Joseph is the lead developer of Vert Studios Follow Joseph on Twitter: @Joe_Query
Subscribe to the blog: RSS
Visit Joseph's site: joequery.me