Flask NoSQL Authentication Tutorial - Part II

Overview

This is the second part of a tutorial that provides instructions for how to create an authentication mechanism for a web application utilizing Flask as the Python web framework and Elasticsearch (ES) as the NoSQL data store.

The first part of the tutorial covered the prerequisites, the Main API, the User model, and the Users API end point. In this second part of the tutorial, I will be covering the Flask-Login and session management modifications required for the main API, the User model, and the Auth API.

Once again, feel free to ask any questions below and I’ll be happy to respond!

Flask-Login

Flask-Login provides user session management for basic authentication tasks; logging a user in and logging out a user, in your application. You can restrict specific views for non-authenticated users by adding a decorator to your view routes. For this tutorial example, I have followed the basic configuration and created a custom user_loader for ES.

Main API

In the Main API, we define the ‘login_manager’ and the ‘load_user’ function for the Flask-Login ‘user_loader’ decorator which sets the callback for reloading a user from the session. The ‘load_user’ funcation creates a User object, checks if the user exists in ES, then returns the User object:

main.pylink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
@login_manager.user_loader
def load_user(email_address):
try:
user = User(email_address=email_address)
except ValueError as error:
message = str(error)
logger.warn(message)
return None
data = {}
try:
data = g.db_client.get('example', user.key)
except (TransportError, Exception) as error:
if not getattr(error, 'status_code', None) == 404:
logger.critical(str(error))
return None
if not data.get('found', None):
message = "'%s' does not exist." % email_address
logger.warn(message)
return None
user.set_values(values=data['_source'])
return user

Then we define the APP_SECRET_KEY as a global variable, then assign it to the main app and instantiate the ‘login_manager’:

1
app.secret_key = APP_SECRET_KEY
login_manager.init_app(app)

That’s all the changes required for the ‘main.py’. We need to modify the User model but those changes are minor too.

User model

For the User model, we need to add a few functions that are required for Flask-Login. The function doc strings should be self explanatory.

User.pylink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def is_authenticated(self):
""" should just return True unless the object represents a user
that should not be allowed to authenticate for some reason.
"""

if self.is_anonymous():
return False
return True

def is_active(self):
""" method should return True for users unless they are inactive, for
example because they have been banned.
"""

if not self.values.get('is_active', False):
return False
return True

def is_anonymous(self):
""" method should return True only for fake users that are not supposed
to log in to the system.
"""

if not self.values.get('is_anonymous', False):
return False
return True

def get_id(self):
""" return the self.key """
return self.values[KEY_NAME]

Auth API

Now for the Auth API, we create a ‘login’ route for authenticating a user and a ‘logout’ for unauthenticating a user. For the ‘login’ route, first, we verify the user submitting the request is valid by checking if the user key exists in ES. Next, we check if the request payload includes the correct password by comparing the password value with the hashed password from the database. Finally, we add the valid user into session via ‘login_user’. The ‘login’ route is almost identical to the ‘new’ user route from the User API, but we add the password check and add the authenticated user via ‘login_user’:

Authlink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
...
logger.debug("'%s' successfully found!", request.json['email_address'])
user.set_values(values=data['_source'])
if not user.check_password(request.json['password']):
logger.warn("'%s' incorrect password", request.json['email_address'])
message = "Unknown email_address or bad password"
return jsonify(message=message, success=False), 400
login_user(user)
message = "'%s' successfully logged in!" % request.json['email_address']
logger.info(message)
...
```

Once a use is authenticated, the active user is now stored in the session. For the 'logout' route, we simply call the 'logout_user()' method to remove the user id from the current session. Now let's create a test route that is only accessible from authorized users.

## Test API

The [Test API](https://github.com/phriscage/flask_elasticsearch_auth_example/blob/master/lib/example/v1/api/test/views.py) includes the 'login_required' decorator which restricts access to only users that are authenticated:

``` Python Test https://github.com/phriscage/flask_elasticsearch_auth_example/blob/master/lib/example/v1/api/test/views.py
...
@test.route('')
@login_required
def index():
...

Import the new auth and test Blueprints and register it with the URL route to the app in main.py:

main.pylink
1
2
3
4
5
6
from example.v1.api.auth.views import auth
app.register_blueprint(auth, url_prefix="/v1/auth")
from example.v1.api.users.views import users
app.register_blueprint(users, url_prefix="/v1/users")
from example.v1.api.test.views import test
app.register_blueprint(test, url_prefix="/v1/test")

Start the application again with the ‘main.py’ and run curl -X GET -D - http://127.0.0.1:8000/v1/test. You should recieve an 401 unauthorized response:

1
$ curl -X GET -D - http://127.0.0.1:8000/v1/test
HTTP/1.0 401 UNAUTHORIZED
Content-Type: application/json
Content-Length: 294
Set-Cookie: session=eyJfaWQiOnsiIGIiOiJOalk0TldVMU1XWXdaamsyT0Roa1pqVmxOamN3TnpRNU5tSmpNamsxTVRJPSJ9fQ.B6pYAg.q2HbuYgeleBAGU1kKfDCCnGEugg; HttpOnly; Path=/
Server: Werkzeug/0.9.6 Python/2.6.6
Date: Tue, 20 Jan 2015 01:18:19 GMT

{
  "error": "401: Unauthorized",
  "message": "The server could not verify that you are authorized to access the URL requested.  You either supplied the wrong credentials (e.g. a bad password), or your browser doesn't understand how to supply the credentials required.",
  "success": false
}

We need to first authenticate our test user, store the cookie, then send the request again. Let’s authenticate the user we created in Part I, ‘test@abc.com’ and store the cookies into a file, ‘cookies.txt’

1
$ curl -X POST -s -D - -c ~/cookies.txt -H 'Content-Type: application/json' -d '{"email_address": "test@abc.com", "password": "test"}' http://127.0.0.1:8000/v1/auth/login
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 360
Set-Cookie: session=eyJfZnJlc2giOnRydWUsIl9pZCI6eyIgYiI6Ik5qWTROV1UxTVdZd1pqazJPRGhrWmpWbE5qY3dOelE1Tm1Kak1qazFNVEk9In0sInVzZXJfaWQiOiJ0ZXN0QGFiYy5jb20ifQ.B58_Qg.Ez4andKJ01l51Ltd5nDg9EyXzTQ; HttpOnly; Path=/
Server: Werkzeug/0.9.6 Python/2.6.6
Date: Tue, 20 Jan 2015 01:22:10 GMT

{
  "data": {
    "_id": "test@abc.com",
    "_index": "example",
    "_source": {
      "_type": "user",
      "created_at": 1417912435.2168,
      "email_address": "test@abc.com",
      "is_active": true
    },
    "_type": "user",
    "_version": 1,
    "found": true
  },
  "message": "'test@abc.com' successfully logged in!",
  "success": true
}

Boom! We’ve successfully authenitcated our test user! You can view the ‘cookies.txt’ to see the current session cookie. Now we can use that session variable to send a request to ‘test’ again: curl -X GET -s -D - -b ~/cookies.txt http://127.0.0.1:8000/v1/test

1
$ curl -X GET -s -D - -b ~/cookies.txt http://127.0.0.1:8000/v1/test
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 273
Set-Cookie: session=eyJfZnJlc2giOnRydWUsIl9pZCI6eyIgYiI6Ik5qWTROV1UxTVdZd1pqazJPRGhrWmpWbE5qY3dOelE1Tm1Kak1qazFNVEk9In0sInVzZXJfaWQiOiJ0ZXN0QGFiYy5jb20ifQ.B58_6Q.JoOanNrX80o0hiBnrwGllvUg1G8; HttpOnly; Path=/
Server: Werkzeug/0.9.6 Python/2.6.6
Date: Tue, 20 Jan 2015 01:24:57 GMT

{
  "data": {
    "cookies": {
      "session": "eyJfZnJlc2giOnRydWUsIl9pZCI6eyIgYiI6Ik5qWTROV1UxTVdZd1pqazJPRGhrWmpWbE5qY3dOelE1Tm1Kak1qazFNVEk9In0sInVzZXJfaWQiOiJ0ZXN0QGFiYy5jb20ifQ.B58_Qg.Ez4andKJ01l51Ltd5nDg9EyXzTQ"
    }
  },
  "message": "Test",
  "success": true
}

That’s it! There’s not alot too it. You can use the ‘login_required’ decorator on any view that requires authentication. There are some session expiration configuration options and custom authentication params that are confgiurable in Flask-Login.

I hope you have found this tutorial helpful and maybe even learned a thing or two about Python, Flask, authentication, etc. Let me know if you have any questions.

Best,

Chris

Flask NoSQL Authentication Tutorial - Part I

Overview

This tutorial provides instructions for how to create an authentication mechanism for a web application utilizing Flask as the Python web framework and Elasticsearch (ES) as the NoSQL data store. Many applications utilize ES as the index/search layer, but I choose ES as the primary database as a proof of concept for both persistant and search data layers. ES can be swapped out with almost any available NoSQL document store.

A basic understanding of the *NIX system, Python, and web applications is required otherwise you may struggle with some of the concepts and context. If you are new to Flask, I highly recommend checking out Miguel Grinberg’s Flask Mega Tutorial or his newley published Flask Book by O’Reilly for a complete Flask application how-to. The User Login tutorial actually inspired me to build this tutorial for a NoSQL data store.

In this first part of the tutorial, I will be covering the prerequisites, the main API, the User model, and the Users API end point. If you have any questions, feel free to write below and I’ll be happy to answer if you have any issues.

Let’s get started!

Prerequisites

Below are the specific prerequisites that are required to setup the working environment and download the neccesary packages and files.

  • linux server: This tutorial is based off the Centos 6.4 x86_64 base image, so package management (and command instructions below) are via RPM and Yum. sudo or root privileges are required to install the various system packages. If you prefer Debian, you’ll need to substitute the respectable DEB packages and apt-get commands.

ssh username@hostname

  • Elasticsearch: The ES server package is downloaded directly from the ES site. Installation and the default configuration is all that is required to get the service running. You can verify ES is running by executing curl -X GET http://127.0.0.1:9200 or navigating to the URL.
    note that version 1.3.2 is used at the time of writing

wget wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.noarch.rpm && yum install elasticsearch-1.3.2.noarch.rpm --nogpgcheck -y
service elasticsearch start

1
$ curl -X GET http://127.0.0.1:9200
{
  "status" : 200,
  "name" : "Sludge",
  "version" : {
    "number" : "1.3.2",
    "build_hash" : "dee175dbe2f254f3f26992f5d7591939aaefd12f",
    "build_timestamp" : "2014-08-13T14:29:30Z",
    "build_snapshot" : false,
    "lucene_version" : "4.9"
  },
  "tagline" : "You Know, for Search"
}
  • Python: Python 2.6.6 is already included in the base Centos 6.4, so that version will work. We’ll be using Python virtual environments and Pip to handle the Python libraries and dependencies:

yum install python-virtualenv python-virtualenvwrapper python-pip -y

  • Git: We’ll need to install Git and clone the tutorial source code from my Gihub repository.

yum install git -y
git clone https://github.com/phriscage/flask_elasticsearch_auth_example && cd flask_elasticsearch_auth_example

  • Python libraries: Create a new virtual environment and activate it. Then pull the packages from PyPi using Pip and requirements.txt:

mkvirtualenv flask_elasticsearch_auth_example -r requirements.txt

Now we should have all the required dependencies. :)

Main API

Before we create the primary User model, we need to create the basic Flask app API and verify we can connect to ES. I’m using Flask’s global g module to handle the ES client connection for each request. You can tweak the ES connection pool options for the cluster, but for now the default connection object works. I am using the default_error_handle method to return a standard JSON formatted message for all of the relevant HTTP error codes.

main.pylink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
def connect_db():
""" connect to couchbase """
try:
db_client = Elasticsearch()
#[{'host': ELASTICSEARCH_HOST, 'port': ELASTICSEARCH_PORT}],
#use_ssl=True,)
#sniff_on_connection_fail=True,)
except Exception as error:
logger.critical(error)
raise
return db_client

def create_app():
""" dynamically create the app """
app = Flask(__name__)
app.config.from_object(__name__)

@app.before_request
def before_request():
""" create the db_client global if it does not exist """
if not hasattr(g, 'db_client'):
g.db_client = connect_db()

def default_error_handle(error=None):
""" create a default json error handle """
return jsonify(error=str(error), message=error.description,
success=False), error.code

## handle all errors with json output
for error in range(400, 420) + range(500, 506):
app.error_handler_spec[None][error] = default_error_handle

The main.py arguments accept a specific hostname or IP and port number. When you start the application, the output should look like this:

1
$ ./main.py
2014-12-06 22:10:05,770 INFO werkzeug[8640] : _log :  * Running on http://0.0.0.0:8000/
2014-12-06 22:10:05,770 INFO werkzeug[8640] : _log :  * Restarting with reloader

We can verify it works, along with the default_error_handle, but pulling the base URL. curl -X GET -D - http://127.0.0.1:8000/

1
$ curl -X GET -D - http://127.0.0.1:8000/
HTTP/1.0 404 NOT FOUND
Content-Type: application/json
Content-Length: 191
Server: Werkzeug/0.9.6 Python/2.6.6
Date: Sun, 07 Dec 2014 00:35:23 GMT

{
  "error": "404: Not Found",
  "message": "The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.",
  "success": false
}

Great! Now let’s define our User model and how-to store the user document data in ES.

User model

The User model contains the data structure and validation methods for the user metadata that will be passed from the API.

First, we include the system level modules and two password hash functions from werkzeug. We define what the key or ID attribute name will be for our user document and any additional required and/or valid attributes for the document.

user.pylink
1
2
3
4
5
6
7
8
9
from __future__ import absolute_import
import time
import re
import logging
from werkzeug.security import generate_password_hash, check_password_hash

KEY_NAME = 'email_address'
REQUIRED_ARGS = (KEY_NAME, 'password',)
VALID_ARGS = REQUIRED_ARGS + ('first_name', 'last_name',)

Instatiation of class executes private class functions to validate the kwargs against the global VALID_AGRS and REQUIRED_ARGS. It also sets the default and required values for the user document:

user.pylink
1
2
3
4
5
6
7
8
9
10
class User(object):
""" encapsulate the user as an object """

def __init__(self, **kwargs):
""" instantiate the class """
self.key = None
self.values = {}
self._validate_args(**kwargs)
self._set_key(kwargs[KEY_NAME])
self._set_values()

The set_password and check_password functions are how the model generates a password hash and verifies a plain text password against a hash. Instead of creating our own hashing algorithms, we use werkzeug’s utilies we imported above:

user.pylink
1
2
3
4
5
6
7
8
9
def set_password(self, password):
""" set the password using werkzeug generate_password_hash """
self.values['password'] = generate_password_hash(password)

def check_password(self, password):
""" check the password using werkzeug check_password_hash """
if not self.values.get('password', None):
return None
return check_password_hash(self.values['password'], password)

There’s not alot going on the User model for Part I, but we will expand the functionality in the next tutorial.

Users API:

Now that we have our basic user model, let’s define the User API endpoint that enables us to create a new user in ES. I’m using Flask’s Blueprint, jsonify, request and g modules. I created a ‘users’ Blueprint and added the root ‘/new’ route to create new users via HTTP POST. REST API Tutorial provides a greate “resource” for learning the appropriate synatx naming. For a truely textbook RESTful interface, one can argue between how a new resource is created ( ‘/users/new’, ‘/user/new’, or ‘/users’) and if resource pluralization matters, but I’ll save that discussion for a later date…

The overall logic is straightforward. First we verify the request content type is ‘application/json’. Next we create the User model and check the payload. Then check if the User document exits in ES. Finally, create a new User document if the User key, email_address, does not exist in ES.

users/views.pylink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)) +
'/../../../../../lib')
from example.v1.lib.user import User, KEY_NAME as USER_KEY_NAME
from flask import Blueprint, jsonify, request, g
from elasticsearch import TransportError
import logging

logger = logging.getLogger(__name__)

users = Blueprint('users', __name__)

@users.route('/new', methods=['POST'])
""" create a user and hash their password

**Example request:**

.. sourcecode:: http

GET /users/new HTTP/1.1
Accept: application/json
data: {
'email_address': 'abc@abc.com',
'password': 'abc123',
'first_name': 'abc',
'last_name': '123'
}

**Example response:**

.. sourcecode:: http

HTTP/1.1 200 OK
Content-Type: application/json

:statuscode 200: success
:statuscode 400: bad data
:statuscode 409: already exists
:statuscode 500: server error
"""


if not request.data:
message = "Content-Type: 'application/json' required"
logger.warn(message)
return jsonify(message=message, success=False), 400
try:
user = User(**request.json)
except ValueError as error:
message = str(error)
logger.warn(message)
return jsonify(message=message, success=False), 400
data = {}
try:
data = g.db_client.get('example', user.key)
except (TransportError, Exception) as error:
if not getattr(error, 'status_code', None) == 404:
logger.critical(str(error))
message = "Something broke... We are looking into it!"
return jsonify(message=message, success=False), 500
if data.get('found', None):
message = "'%s' already exists." % user.values[USER_KEY_NAME]
logger.warn(message)
return jsonify(message=message, success=False), 409
try:
args = {
'index': 'example',
'id': user.key,
'body': user.values,
'doc_type': user.values['_type']
}
data = g.db_client.index(**args)
except Exception as error:
message = str(error)
logger.warn(message)
return jsonify(message=message, success=False), 500
message = "'%s' added successfully!" % user.values[USER_KEY_NAME]
logger.debug(message)
return jsonify(message=message, success=True), 200

Next we need to import the users Blueprint and register it with the URL route to the app in main.py:

main.pylink
1
2
from example.v1.api.users.views import users
app.register_blueprint(users, url_prefix="/v1/users")

If your ‘main.py’ file is not running, restart it. Finally, let’s test creating a new user ‘test@abc.com’ against the Users API with the curl -X POST -H 'Content-Type: application/json' -d '{"email_address": "test@abc.com", "password": "test"}' http://127.0.0.1:8000/v1/users/new

1
$ curl -X POST  -D - -H 'Content-Type: application/json' -d '{"email_address": "test@abc.com", "password": "test"}' http://127.0.0.1:8000/v1/users/new
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 73
Server: Werkzeug/0.9.6 Python/2.6.6
Date: Sun, 07 Dec 2014 00:33:55 GMT

{
  "message": "'test@abc.com' added successfully!",
  "success": true
}

Success!

You’ll notice that if we try to add the same user again, we recieve a 409 conflict error:

1
$ curl -X POST  -D - -H 'Content-Type: application/json' -d '{"email_address": "test@abc.com", "password": "test"}' http://127.0.0.1:8000/v1/users/new
HTTP/1.0 409 CONFLICT
Content-Type: application/json
Content-Length: 70
Server: Werkzeug/0.9.6 Python/2.6.6
Date: Sun, 07 Dec 2014 00:34:01 GMT

{
  "message": "'test@abc.com' already exists.",
  "success": false
}

That’s it for Part I. I’ll follow up in a couple weeks with Part II which will utilize Flask-Login to handle the user session managment.

Best,
Chris