Buildout

GroupServer uses Buildout for building (both production and development), retrieving code, and managing development. If you have never used Buildout before, this 15 minute video provides an introduction. Documentation is at http://www.buildout.org/docs/index.html.

To use Buildout you will need setuptools. On Ubuntu, install setuptools via

sudo apt-get install python-setuptools

Files

buildout.cfg

The main config file for Buildout. This is what Buildout looks to for directions. In GroupServer, buildout.cfg indicates what core eggs should be installed from GroupServer and from third parties. It also indicates which configuration sections should run, and defines a few.

custom.cfg

Refers to additional eggs developed for projects downstream from GroupServer. This which custom eggs should be installed, as well as which versions of those eggs. In general, anything written specifcally for E-Democracy will be referred to in this file.

develop.cfg

Contains a list that tells Buildout how to retrieve the source for an egg. Every egg from GroupServer and E-Democracy should have an entry in here indicating the address to that egg's repository, and version control tool used on that repository.

Technically, this isn't a Buildout file. It is a file used by [Mr. Developer https://pypi.python.org/pypi/mr.developer], a Buildout extension.

versions.cfg

This controls which version of a core egg Buildout will retrieve and build. Most core eggs will have an entry in this file that pins the egg to some version. Some 3rd party eggs will also have their versions pinned by this file.

Issue Tracking - Redmine

Don’t worry if you get a bad certificate warning. That's normal.

Issue/bug tracking, and a place to put ideas for the future. Register for an account, then bug techteam@e-democracy.org to OK your registration.

E-Democracy Redmine -https://redmine.iopen.net/projects/edem

GroupServer Redmine - https://redmine.iopen.net/projects/groupserver

Repository - RhodeCode

https://source.iopen.net/groupserver - don’t worry if you get a bad certificate warning.

A mercurial repository where development code and documentation lives, can be browsed and downloaded from, and where Buildout pulls development code from. Setting up Mercurial

In ~/.hgrc, add the following:

[ui]

username = USERNAME <EMAILADDRESS>

[hostfingerprints]

source.iopen.net = CE:CC:DF:DF:E6:32:89:E3:8A:CD:0B:A1:8F:15:FC:B7:0A:AC:23:D3

Checking Out Eggs

This will checkout an egg into the src directory. Once checked out, changes to the files in src/<egg> will be reflected in the instance running on your machine. Generally, template changes are reflected immediately, while python changes will require a restart of uwsgi.

Checked out eggs under src/ are part of a Mercurial project. If eggs have been checkout for a while, you will need to do am hg pull on them before further development or pushing changes. Also, eggs under src/ are not updated when ./bin/buildout is called.

Step 0: Add ‘develop.cfg’ to the extends line in the [buildout] section of buildout.cfg

Step 1: ./bin/develop checkout <EGG NAME>

Step 2: ./bin/buildout -N

Step 3: Restart uwsgi

Pushing and Pulling Checked Out Code

This is pushing and pulling from Rhode Code or Github, which is the development repo, not the repo from which eggs are pulled during a buildout.

This is simply interacting with Mercurial or Git. If you’ve ever used Mercurial or Git before, you already know this.

Pull

Step 1: cd src/<EGG NAME>

Step 2: hg pull -or- git pull

Push

Step 0: Make sure you have an account on RhodeCode

Step 1: hg add <ANY NEW FILES>

Step 2: hg commit

Step 3: hg push (you’ll be asked for your RhodeCode username/password)

Updating the Egg Repo

http://eggs.iopen.net/groupserver/base/ and http://eggs.iopen.net/groupserver/custom/

This causes the creation/updating of eggs on the egg repo - which is what Buildout pulls from.

Step 0: Have ./bin/fab and ./fabfile.py

Step 1: cd MYBUILDDIR

Step 2: ./bin/fab deployegg:THE.COMPONENT.NAME,custom

Only put the ,custom there if it is an egg specific to a customer ... in your case that would be if it is an egg that has 'edem' anywhere in the name. Otherwise, leave it off.

Step 3: If you’re planning to deploy this egg take note of the version string it produces, as you’ll need that for deploying new/updated eggs into installations.

Figuring out Egg Version Numbers

The format of version strings is <version>-<datetime>-<hg/git hash>

<version> is simply the version of the egg as defined in version.py

<datetime> is the UTC date and time that the egg was deployed, in YYYYMMDDHHMMSS.

<hg hash> is the hash generated for the repo version that the egg version represents. It will be the portion of the hg version string seen on the changelog page that follows the colon (:).

Dev Server

dev.forums.e-democracy.org

DB is updated regularly via cron based on a backup of the live db

can ssh in: deploy@ or root@

dev.forums.e-democracy.org needs to have your public key

Deploying eggs to this server

git pull

./bin/buildout -N

./reload.sh

Production Deployment

Just email version strings to codedeploy@onlinegroups.net .

Send this email by Friday night (central time).

Creating a new egg

See also: http://groupserver.org/groups/development/guidelines/eggs

Make a directory in src for the egg

Copy the following files from an existing egg and edit as needed, or create these files while using the files in another egg as a reference

The docs directory

MANIFEST.in

README.txt

Edit to be blankish, with a brief description of what the egg does.

setup.cfg

setup.py

Four things need to change: 1. The name needs to be the same as your egg, 2. The description should be sane, 3. The namespace_packages need to reflect the hierarchy. For example, in the case of edem.group.messages.topics, namespace_packages should be set to "['edem', 'edem.group', 'edem.group.messages']". (The final directory is supposed to be missing; it is a normal package, not a namespace package.) 4. Set the install_requires to just 'setuptools' for now.

version.py

.hgignore

Next copy an __init__.py from a namespace-directory of the original egg.

In Python the ability to write "from package.package.package import SomeThing" is not built into the language. It is a hack. The hack is called a "namespace package". The hack is in two parts. First, there is a "namespace_packages" entry in the setup.py. Second there is a special __init__.py in all the namespace packages. Copy this special __init__.py from *any* other package to each of the directories in the namespace.

Next you need a __init__.py in the actual package (module) directory.

Just enter the following into the module directory __init__.py

1. coding=utf-8
2. This space deliberately left blank

Zope stuff! Add a configure.zcml to your module directory. To start with, enter the following:

          <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
          <configure xmlns="http://namespaces.zope.org/zope"
            xmlns:browser="http://namespaces.zope.org/browser"
            xmlns:five="http://namespaces.zope.org/five"
            i18n_domain="groupserver">
          </configure>

Init a repo and commit:

1. Change to the top folder of the egg 2. Run Mercurial:

   1. hg init .
   2. hg addremove
    3. hg ci -m"Initial import."

Finally, weave your new egg (which does nothing) into GroupServer.

Open "custom.cfg" in your top-level the GroupServer directory.

Add "PACKAGENAME" to the "custom-zope-eggs" list.

Open "develop.cfg" in your top-level the GroupServer directory.

Add "PACKAGENAME = fs PACKAGENAME" [all on one line] to the "sources" list.

Turn on the egg by running Mr Developer in your GroupServer directory: "./bin/develop a PACKAGENAME"

Rebuild.

Start GroupServer.

And you should have a new egg.

Estimating Project Time Requirements

See https://redmine.iopen.net/projects/groupserver/wiki/TimeEstimation