JADDOG

A screencast I created  has landed on youtube. It demonstrates using audrey to launch a multi-instance deployment in EC2 using the Aeolus Project.

More info at
https://www.aeolusproject.org/
https://www.aeolusproject.org/audrey.html

I’ve spent the past 3 years developing two django projects.
Nushus: https://fedorahosted.org/nushus
Loki: https://fedorahosted.org/loki

When Red Hat released openshift I was interested to deploy django on it. I use MySQL with my django apps, mainly because that’s what I know well. The tutorial I was working through used sqlite so I’ve put this together to show how I got MySQL working with django on openshift.

Run this quick start to get your account and domain setup
https://openshift.redhat.com/app/express#quickstart
Then login to the website and open this turoial:
https://www.redhat.com/openshift/kb/kb-e1010-show-me-your-django-getting-django-up-and-running-in-5-minutes
Run the ‘Deploying a Django Application’ section of this tutorial to get a basic django app setup and running. Then you’ll have Django setup but with no admin:

After you get here, continue the tutorial related to the usual code edits needed in your django project to enable the admin. Don’t setup the sqlite database as the tutorial suggests, we’ll get connected to MySQL next.
Next lets setup the database:

testapp git:(master)➤ rhc-ctl-app -e add-mysql-5.1 -a testapp
Password:
Contacting https://openshift.redhat.com
Contacting https://openshift.redhat.com

RESULT:

Mysql 5.1 database added.  Please make note of these credentials:

   Root User: SuperSecretUser
   Root Password: SuperSecretPassword

Connection URL: mysql://127.XXX.XXX.XXXX:3306/

This will create the MySQL instance but not the database.
This forum post suggests the current method to get the database setup: https://www.redhat.com/openshift/forums/express/mysql-db-name
We need to also run syncdb, which the django tuorial shows us how to do. It also shows us how to deploy the admin media. I don’t like how the django tutorial makes us commit static code that’s already deployed. Let’s copy it remotely from the egg that’s already deployed into the remote static dir without commiting it. All this is done via the openshift build hook. I’ve rewritten it a bit in process of testing things so it will look a bit different than the tutorials:

testapp git:(master)➤ cat .openshift/action_hooks/build
#!/bin/bash
# This is a simple build script, place your post-deploy but pre-start commands
# in this script.  This script gets executed directly, so it could be python,
# php, ruby, etc.

# create the database if it doesn't exist
# https://www.redhat.com/openshift/forums/express/mysql-db-name
if ! /usr/bin/mysql -u "$OPENSHIFT_DB_USERNAME" --password="$OPENSHIFT_DB_PASSWORD" -h "$OPENSHIFT_DB_HOST" -e "show tables;" $OPENSHIFT_APP_NAME > /dev/null
then
    /usr/bin/mysqladmin -u "$OPENSHIFT_DB_USERNAME" --password="$OPENSHIFT_DB_PASSWORD" -h "$OPENSHIFT_DB_HOST" create "$OPENSHIFT_APP_NAME"
    echo "Created MySQL database $OPENSHIFT_APP_NAME"
fi

# copy the admin media into place
if [ ! -d "$DIRECTORY" ]; then
    if mkdir $OPENSHIFT_REPO_DIR/wsgi/static/admin
    then
        echo "Created directory $OPENSHIFT_REPO_DIR/wsgi/static/admin"
        echo "Copying admin media into $OPENSHIFT_REPO_DIR/wsgi/static/admin"
        cp -R $OPENSHIFT_APP_DIR/virtenv/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/contrib/admin/media/** $OPENSHIFT_REPO_DIR/wsgi/static/admin
    fi
fi

# cd into the project to run django manage.py commands
cd $OPENSHIFT_REPO_DIR/wsgi/testapp

# run syncdb
# https://www.redhat.com/openshift/kb/kb-e1010-show-me-your-django-getting-django-up-and-running-in-5-minutes
echo "Executing './manage.py syncdb --noinput'"
./manage.py syncdb --noinput

This gets us to the login page for the admin:

Now all that’s missing is a user. I toyed with a couple options to get a basic user in. The command line manage.py won’t let you pass a password and I had trouble getting fixtures to load. Though, both of those required sql to verify if the user existed. Settled on a simple management command to make sure we have a user. We have to put it in an app, but we’re going to need an app eventually anyways to make django so more than start. So here’s what I did.
In the project directory run the django startapp command:

testapp/wsgi/testapp git:(master+)➤ ../manage.py startapp myapp

Then add it to the INSTALLED_APPS in your settings file

+     'testapp.myapp',

You have to put the projects name in there, otherwise things won’t work later. I got 500′s trying just to put just ‘myapp’ in the installed apps. This app won’t really do anything for now. It’s just a container for our management command. You can make it into something else later. Next create the directory structure for the command in the myapp directory.

testapp/wsgi/testapp git:(master+)➤ mkdir -p myapp/management/commands
testapp/wsgi/testapp git:(master+)➤ touch myapp/management/__init__.py myapp/management/commands/__init__.py
testapp/wsgi/testapp git:(master+)➤ vim myapp/management/commands/ensuresuperuser.py
testapp/wsgi/testapp git:(master+)➤ tree myapp
myapp
|-- __init__.py
|-- management
|   |-- commands
|   |   |-- ensuresuperuser.py
|   |   `-- __init__.py
|   `-- __init__.py
|-- models.py
|-- tests.py
`-- views.py

2 directories, 7 files

You can see I called my command ensuresuperuser. The command will check if the user exists and create it with a password equal to the user’s username if the user doesn’t exist. Here’s it’s code:

testapp/wsgi/testapp git:(master+)➤ cat myapp/management/commands/ensuresuperuser.py
from django.core.management.base import BaseCommand, CommandError
from django.contrib.auth.models import User

class Command(BaseCommand):
    args = 'username'
    help = 'make sure a user exists with a password'

    def handle(self, *args, **options):
        try:
            user = User.objects.get(username=args[0])
        except:
            User.objects.create_superuser(args[0], email=args[0]+'@example.com', password=args[0])
            self.stdout.write('user %s created with password %s\n' % (args[0], args[0]))

Last thing to do is to add this command to your build hook so it executes when you push your code, so update your build hook. Here’s the whole contents of mine:

testapp git:(master+)➤ cat .openshift/action_hooks/build
#!/bin/bash
# This is a simple build script, place your post-deploy but pre-start commands
# in this script.  This script gets executed directly, so it could be python,
# php, ruby, etc.

# create the database if it doesn't exist
# https://www.redhat.com/openshift/forums/express/mysql-db-name
if ! /usr/bin/mysql -u "$OPENSHIFT_DB_USERNAME" --password="$OPENSHIFT_DB_PASSWORD" -h "$OPENSHIFT_DB_HOST" -e "show tables;" $OPENSHIFT_APP_NAME > /dev/null
then
    /usr/bin/mysqladmin -u "$OPENSHIFT_DB_USERNAME" --password="$OPENSHIFT_DB_PASSWORD" -h "$OPENSHIFT_DB_HOST" create "$OPENSHIFT_APP_NAME"
    echo "Created MySQL database $OPENSHIFT_APP_NAME"
fi

# copy the admin media into place
if [ ! -d "$DIRECTORY" ]; then
    if mkdir $OPENSHIFT_REPO_DIR/wsgi/static/admin
    then
        echo "Created directory $OPENSHIFT_REPO_DIR/wsgi/static/admin"
        echo "Copying admin media into $OPENSHIFT_REPO_DIR/wsgi/static/admin"
        cp -R $OPENSHIFT_APP_DIR/virtenv/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/contrib/admin/media/** $OPENSHIFT_REPO_DIR/wsgi/static/admin
    fi
fi

# cd into the project to run django manage.py commands
cd $OPENSHIFT_REPO_DIR/wsgi/testapp

# run syncdb
# https://www.redhat.com/openshift/kb/kb-e1010-show-me-your-django-getting-django-up-and-running-in-5-minutes
echo "Executing './manage.py syncdb --noinput'"
./manage.py syncdb --noinput

# add an admin user
./manage.py ensuresuperuser admin

This time when you commit and push everything you will see it tell you “user admin created with password admin”. Obviously this is really insecure so go ahead and change that password right away. Wouldn’t want your throw away testapp become something it wasn’t intended for.

There you have it, OpenShift + Django + MySQL.
It’s worth noting, If you get a 500 error you need to use the rhc-snapshot command like so:

testapp git:(master+)➤ rhc-snapshot -a testapp
Password:
Contacting https://openshift.redhat.com
Pulling down a snapshot to testapp.tar.gz

This pulls down a tar ball of your running environment and has a logs directory in there that you can look at the apache logs. I’ll second Ian’s suggestion for a rhc-logwatch command. That would be pretty handy.

Next steps for me:
1. How to delete an OpenShift application (i.e. throw away testapp, I’m done with it)
2. Install Nushus in OpenShift!

*** Update ***
I came across another blog post a day later that reference rhc-tail-files. i.e. the watchlog thing I mentioned above is already a feature:

testapp git:(master+)➤ rhc-tail-files -a testapp
Password:
Contacting https://openshift.redhat.com
Attempting to tail files: testapp/logs/*
Use ctl + c to stop

==> testapp/logs/access_log-20110817-000000-EST <==
... snip log files output ...

I’ve been using Fedora 15 and Gnome 3 for a little over  2 months now. I’ve learned a few things that have made my daily workflow a little easier, thought I’d share.

1. the alt key switches the Suspend menu item to “Power Off”
I infrequently need it, but the alternative was to logout and power off from the login screen or to add a gnome-shell-extension.

2. gnome-tweak-tool
There’s a couple settings in there that I was glad to be able to tweak

3. Drag to top of the screen to maximize.
I keep a couple things maximized, It’s nice to just drag the window to the top of the screen and have it maximize.

4. gsettings
I haven’t had too much need for this yet, but understanding it is relevant to writing extensions.

5. gnome-shell-extensions-dock
yum install gnome-shell-extensions-dock
gsettings set org.gnome.shell.extensions.dock position left
alt-f2
r
enter

6. wrote a gnome-shell-extension

Part of my team at work is in Pune, India and our company also pass lots of times around in utc. The Fedora 14 Clock applet that listed what time it was in other timezones was helpful. So I set out last week to put something together in gnome 3 that would serve the same purpose.

In the process I also found some code to add apps to the top panel and decided to post the little bit of code I put together here: https://github.com/radez/gnome-shell-extensions

Here’s what my “clocks” extension looks like. It’s not much but it suits my needs. I need to plug into gsettings as some point so that Pune and UTC arn’t hard coded.

At work I use Loki to manage my buildbot infrastructure. It’s deployed on apache via mod_wsgi. Recenlty I’ve been having trouble with Apache crashing with one of two single error lines in the logs:

[notice] seg fault or similar nasty error detected in the parent process
or
[emerg] (28)No space left on device: Couldn’t create accept lock

I came across this post and found it very helpful. Start with seeing if there are left over semaphores when you stop apache:

# ipcs -s | grep apache

If that’s the case then first clear them:

# /usr/bin/ipcs -s | grep apache | awk ‘ { print $2 } ‘ | xargs ipcrm sem

Then tell the kernel to allow more semaphores by adding the followings lines to /etc/sysctl.conf

kernel.msgmni = 1024
kernel.sem = 250 256000 32 1024

then run

# sysctl -p

Once I start everything back up this seems to have fixed my issues.

New version of nushus is available for download.
Couple bug fixes and a couple small features.
New cli plugin added as part of the standard distribution.

https://fedorahosted.org/nushus
Release Notes: https://fedorahosted.org/nushus/wiki/Release

Loki 0.10.1
https://fedorahosted.org/loki

Just put a tarball out on the loki downloads page: https://fedorahosted.org/released/loki/

Check out the change log here: https://fedorahosted.org/loki/wiki/ChangeLog

I’m currently working on consuming some of the yum-utils package on Fedora/RHEL to add a repo “mirror” sort of functionality to nushus. One of the files in the package is /usr/bin/repodiff which defines a class that has some functionality that would be super helpful to not have to rewrite. Problem is that you can’t do a straight import on that file because it doesn’t have a .py extension. A little googling for variants of “python import non .py” had a couple different ways to do the import. Best answer I’ve found that provides access to the class was found on a little cheat sheet. Just wanted to post a link and a copy, it has some good information for those new to python and a few little gems of “things you don’t do all the time”.

the link on google went directly to the pdf: All I ever needed to know about python scripting
Here’s my copy of it: All I ever needed to know about python scripting

Answer to my question is on page one of the pdf:

>>> from types import ModuleType
>>> my_script = ModuleType(‘my_script’)
>>> exec open(‘bin/my_script’) in my_script.__dict__
>>> my_script.main(['-t', 'bar'])

Thanks Matt Harrison! (no idea who you are, but thanks!)
Maybe I’ll email him now

In case anyone is looking to learn python, here’s the resource I learned on: Dive into Python

Rewrote the model in loki to separate builders from slaves in attempt to support multinode support. I’ve also submitted a package request for Django-loki to fedora.

https://fedorahosted.org/loki

I got some feedback yesterday that suggested the Nushus Fedorahosted.org page could mistakenly depict the application as CLI only. The feedback also suggested that some screenshots would be a welcome addition to the trac site and would help remedy this mis-perception.

So here the are: some nushus screenshots

New version of nushus is available for download. Just a couple bug fixes.

https://fedorahosted.org/nushus
Release Notes: https://fedorahosted.org/nushus/wiki/Release