E-Book Enlightenment

A Booki Of Your Own

Reasons For Having Your Own Booki

e_books.png

For most it will not be necessary to have their own Booki software, and every reason not to.  Setting up your own Booki and backing up the data on a regular basis is a significant amount of work, and if you're going to share your work with the world there is really no reason not to use the shared Booki as well.

If you're not going to share, then having your own Booki makes sense.  If you don't have reliable access to the Internet having your own Booki might make sense as well.  A school with XO laptops that can connect to each other on a network but not to the Internet might find a local copy of Booki quite valuable.  In my own case I set up Booki on a computer at home and another at the office.

My day job involves working on an application framework and teaching people how to use it.  An application framework is to software what a chassis is to a car, more or less, and just as a factory can build many different cars on the same chassis you can have many applications using the same application framework.  Teaching people to use this framework is a challenge, and teaching people who live and work on the other side of the Earth is a greater challenge.  We had been using articles on a website, plus a Wiki, to contain the training materials, but after writing two FLOSS Manuals I came to the conclusion that what my company needed was an honest to gosh Manual.

I installed the software at home because:

  • I wanted the work install to go as quickly and smoothly as possible
  • I want, eventually, to write a book that I will not share with the world.  After self-publishing my first FLOSS Manual on Lulu I honestly felt that it would be easier to use Booki to lay out this book (working title: Jim's Oprah Book) than to do the same thing with Open Office or MS Word.

If your reasons are like mine, then let's set up our own Booki!

Getting The Software

You will need a recent version of Linux to run Booki on.  Windows or a Mac will not run Booki.  You won't need much of a computer to run it on.  The computer I used at home was a refurbished IBM NetVista which I had bought online for about a hundred dollars.  The computer at work was a discarded desktop model which was even older.  I would not recommend trying to install this software on an XO laptop, but any desktop computer made in the last few years should be fine.

I used Fedora 13 for these Booki installs, but I don't recommend it.  Fedora is used on the XO laptops, and since I write software for that platform I use it on my desktop computers as well.  The downside of Fedora is that, so far, every time I've upgraded to a new version of Fedora I've had to back up all my data and do a complete reinstall.  Other than this, the different brands of Linux are more or less the same.  If I was going to recommend a Linux for Booki my choices in order would be:

  • Whatever Linux you already have
  • Ubuntu.  If you've never used Linux, this is probably the easiest, and I have had good experiences with it.

The rest of these instructions will assume that you have Linux installed and have become comfortable running it.  Getting comfortable with Linux is the subject for another book, so if you've never used Linux it would be a good idea to find someone who has to help.

Often the install program for Linux will ask if you intend to use it as a web server or if you are doing programming or if you want office software.  Answering "Yes" to all three will save some time.

There are actually two parts to Booki, and you'll need both:

  • Booki itself
  • OBJAVI 2, the part that creates PDF's, EPUBs, etc from your Book.

installing most software on Linux is no more difficult than checking a check box on an Add/Remove Software dialog, but when software is still under development like Booki is you'll need to get the source code and work with that.  The source code for both is kept in a Git repository, so you'll need to have Git installed.  Once you do, you can create a "src" directory in your home directory and from in that directory run these commands:

git clone git://booki-dev.flossmanuals.net/git/booki.git
git clone git://repo.or.cz/objavi2.git

This will create two directories under "src": booki and objavi2, which will contain the source code for these products.

Before you can continue, you'll need to check the README.txt file in Booki and the INSTALL file in OBJAVI 2 to find out what other software you'll need to install.  For Booki the list is:

  • django
  • mod_python
  • apache2
  • php5
  • python-simplejson
  • sqlite
  • redis
  • aspell plus dictionaries for the languages you will use.

If you're lucky the latest redis will be included as a package in your distribution.  Fedora users so far are not so lucky.  They will need to download the latest source code from http://code.google.com/p/redis/ and compile it using the Makefile in the time honored manner:

make
sudo make install

If you installed it from a package in your distribution you should check the Services to see that it is enabled and running.  If you compiled from source you can start it up like this (running as root):

redis-server &

You'll want to run this command every time your computer boots up.  For Fedora 13 you can put this command in /etc/rc.d/rc.local.

For OBJAVI the list is:

  • lxml
  • pdfedit (4.1+)
  • xvfb
  • fontconfig
  • pdftk
  • psutils
  • poppler-utils or xpdf-utils
  • wkhtmltopdf
  • open office 3
  • some fonts

You can get everything from your distribution's packages except wkhtmltopdf.  That you'll need to download from http://code.google.com/p/wkhtmltopdf/.  You should get a precompiled binary and copy it to the /usr/local/bin directory as user root.

When you set up Linux you should get fonts as part of the basic install, but it would be a good idea to install more.  In Fedora the Add/Remove Programs dialog has a section for Fonts that has over a hundred free fonts that you can install.  If you use the Roman alphabet only you can limit yourself to the Latin fonts.

Disable SELinux

Fedora 12 and later has SELinux.  SELinux is to Linux is as Aunt Polly is to Huckleberry Finn.  Its job is to keep programs, including the Apache Web Server, from doing things they should not.  Even without SELinux Linux is pretty secure.  The web server runs as a user (in Fedora's case the user is named "apache") and it is only allowed to do what that user is allowed to do.  On a personal or corporate network this is generally enough.

What SELinux does is add an extra layer of protection.  Programs are expected to do certain things.  If a program tries to do something that SELinux is not expecting, then SELinux stops it.  To get around this you have to tell SELinux to expect this behavior from this specific program.  Then it will be allowed.  This extra layer of protection makes it more difficult for a malicious programmer to break the system.  Sooner or later he will have to do something SELinux is not expecting.  He will be stopped and his actions will be logged.

As commendable as this is, if you have a program that does more than a few unusual things SELinux will be a real challenge.  OBJAVI falls into that category.  It would be a great deal of work to get SELinux to tolerate all the things that OBJAVI is likely to do, and if you're running it on a private network there wouldn't be much benefit.  We disable SELinux by editing a file /etc/selinux/config as user root.  The file should look like this:

 # This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing - SELinux security policy is enforced.
#       permissive - SELinux prints warnings instead of enforcing.
#       disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#       targeted - Targeted processes are protected,
#       mls - Multi Level Security protection.
SELINUXTYPE=targeted 

Change the value of SELINUX to disabled, save the file and reboot your computer.

Configure Booki and OBJAVI 2

If you've ever installed anything written in Python on a Linux system you're familiar with the setup.py program, which copies the Python code where it needs to go in the system.  Eventually both Booki and OBJAVI will have these, but since they are still being developed they don't have them yet.  We will be running the programs out of our home directory for now.

The configuration file for Booki is booki/lib/booki/settings.py.  There is a file named settings.py.original which you need to copy to the name settings.py.  There are several places you'll need to modify in this file.  First is the doc root:

STATIC_DOC_ROOT = '/home/jim/src/booki/site_media'

This points to the site_media directory in the Booki source you got from Git.  You'll need to make certain that this directory has a group ownership of apache (or whatever user apache runs as in your distribution) and that the group is allowed to read and create files in this directory.  Next you need to set up some URL's:

# use this objavi server
OBJAVI_URL = "http://127.0.0.1/objavi.cgi"
ESPRI_URL = "http://127.0.0.1/espri.cgi"
TWIKI_GATEWAY_URL = "http://127.0.0.1/booki-twiki-gateway.cgi"

#the name of the booki server (comment out to use os.environ['HTTP_HOST'])
THIS_BOOKI_SERVER = '127.0.0.1:8000'

IP Address 127.0.0.1 is of course the localhost IP address.  It is likely that you will want to change this to the IP address of your computer so that you can use Booki on the network.  You can of course use a DNS name rather than an IP address.

Notice that we have Booki running on port 8000.  We need to set up virtual hosts for both Booki and OBJAVI.  If you can give each one its own IP address or DNS name there is no reason you can't run both on port 80 like a normal web application.  If everything has to use the same IP address then you can distinguish your virtual hosts from each other using a port number.  Note that OBJAVI has to run on port 80, but Booki can use any port.  There is nothing magic about the number 8000.  It just needs to be a port that nothing else is using.  In the office I use port 86.

Next we have to set up the database entries:

DATABASE_ENGINE = 'sqlite3'
# DATABASE_NAME = 'booki'             # Or path to database file if using sqlite3.
DATABASE_NAME = '/home/jim/booki/booki.db'             # Or path to database file if using sqlite3.
DATABASE_USER = ''             # Not used with sqlite3.
DATABASE_PASSWORD = ''         # Not used with sqlite3.
DATABASE_HOST = 'localhost'             # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = ''             # Set to empty string for default. Not used with sqlite3.

Booki can be used with Sqlite3 or Postgres.  If you are willing to tinker with the code you could get it to run with just about any database.  Sqlite3 is the easiest to set up and is completely adequate for Booki on a small private network.  The only entry you need to touch is DATABASE_NAME, which is set to put the database in a directory that can be read and written to by user apache.  There is no need to create the database file.  Booki will create it.

Now from the command line, as yourself, run this:

export PYTHONPATH=$PYTHONPATH:/home/jim/booki/lib

where /home/jim is your own home directory.  Then change to the booki/lib/booki directory and run this:

./manage.py syncdb

This will create your database tables.  Now you can run Booki itself and check it out.  First run this as yourself:

./manage.py runserver

Once this is running you should be able to point your web browser to http://127.0.0.1:8000 and see Booki in operation.  You should also go to http://127.0.0.1:8000/admin and set up an admin user, plus you'll need to add a license like this:

Django Admin

You can fool around with Booki some more if you like, but be aware that this is not an adequate way to run Booki.  You really need to run it under a virtual host in Apache.  manage.py only supports one user at a time, and is only good as a quick sanity check to make sure everything is set up correctly.  When you're done looking at Booki you can kill manage.py by pressing Ctrl-C in the terminal where you started it.

Next we need to configure OBJAVI.  OBJAVI has its own configuration file, objavi2/objavi/config.py.  You'll need to change some settings in this file, create some new directories, and make the directories readable and writable by the apache group.  The first setting to change is this one:

WKHTMLTOPDF = '/usr/local/bin/wkhtmltopdf'

When you downloaded the binary for wkhtmltopdf it may have had a name like wkhtmltopdf-static or something else.  Change the value of WKHTMLTOPDF to whatever name it has in /usr/local/bin.  (In my case I had renamed it to wkhtmltopdf, which hurt nothing).

The biggest change is to add an entry to SERVER_DEFAULTS.  The first entry for '127.0.0.1:8000' is my own new entry.  Use the actual IP address of your machine if you're going to use it over the network. 

SERVER_DEFAULTS = {
    '127.0.0.1:8000': {
        'css-book': '/static/simmons.css',
        'css-web': '/static/en.flossmanuals.net-web.css',
        'css-newspaper': '/static/en.flossmanuals.net-newspaper.css',
        'css-openoffice': '/static/en.flossmanuals.net-openoffice.css',
        'lang': 'en',
        'dir': 'LTR',
        'toc-encoding': None,
        'display': True,
        'interface': 'Booki',
        'toc_header': 'Table of Contents',
        },
    'booki.flossmanuals.net': {
        'css-book': '/static/en.flossmanuals.net.css',
        'css-web': '/static/en.flossmanuals.net-web.css',
        'css-newspaper': '/static/en.flossmanuals.net-newspaper.css',
        'css-openoffice': '/static/en.flossmanuals.net-openoffice.css',
        'lang': 'en',
        'dir': 'LTR',
        'toc-encoding': None,
        'display': False,
        'interface': 'Booki',
        'toc_header': 'Table of Contents',
        },
}

The entries that I changed are in bold.  I use my own style sheet when creating PDFs for books, and you may wish to do that too.  I indicate that I want to display this server in OBJAVI's list of servers and that I don't want to display the other entries.  You'll see why in a minute.

You need to create some directories under objavi2/htdocs and make certain they can be read and written to by the apache group:

  • books
  • booki-books
  • progress
  • tmp

You'll also need to make certain that the static directory already in objavi2/htdocs is writable by the apache group.  Finally, you'll need to set up a log directory under objavi2 and make sure that apache can create files there too.

Setting Up Apache Virtual Hosts

The simplest way to get Booki and OBJAVI 2 running under the Apache web server is to set up virtual hosts.  What I did was to edit the /etc/httpd/conf/httpd.conf file as the root user using gedit.  It is also possible to make configuration files outside of httpd.conf that will be loaded by Apache automatically.  For Fedora 13 you could make separate files for each virtual host and put them in directory /etc/httpd/conf.d.  (When I got everything working on my Booki install I moved the virtual host entries to files named booki_vh.conf and objavi_vh.conf respectively).

The entries I put at the end of httpd.conf looked like this:

Listen 8000

<VirtualHost *:8000>
        ServerAdmin nicestep@gmail.com

        DocumentRoot /home/jim/src/booki/

        <Directory /usr/share/pyshared/django/contrib/admin/media/>
        Order allow,deny
        Allow from all
        </Directory>

        <Location "/">

            SetHandler python-program
            PythonHandler django.core.handlers.modpython
            SetEnv DJANGO_SETTINGS_MODULE booki.settings
            PythonDebug On
            PythonPath "['/home/jim/src/booki/lib'] + sys.path"

        </Location>

        <Location "/favicon.ico">
            SetHandler None
        </Location>
        <Location "/media">
            SetHandler None
        </Location>
        <Location "/site_media/xinha/plugins/SpellChecker/spell-check-logic.php">
            SetHandler application/x-httpd-php
        </Location>
        <Location "/site_media/xinha/plugins/SpellChecker/spell-check-savedicts.php">
            SetHandler application/x-httpd-php
        </Location>
        <Location "/site_media/xinha/plugins/SpellChecker/aspell-setup.php">
            SetHandler application/x-httpd-php
        </Location>

        Alias /favicon.ico /home/jim/src/booki/favicon.ico
        Alias /media/ /usr/share/pyshared/django/contrib/admin/media/

        ErrorLog /var/log/apache2/booki-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/booki-access.log combined
        ServerSignature On

</VirtualHost>

<VirtualHost *:80>
        ServerAdmin nicestep@gmail.com
        #XXX set server name
        #ServerName objavi.halo.gen.nz
        # limit MEM to 800 million bytes
        RLimitMEM 800000000

        #Sometimes it takes a while. Wait.
        TimeOut 600

        DocumentRoot /home/jim/src/objavi2/htdocs
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /home/jim/olpc/objavi2/>
                Options +All +ExecCGI
                AllowOverride None
                Order allow,deny
                Allow from all
                AddHandler cgi-script .cgi
                # Remove output filters in case mod_deflate is being used.
                RemoveOutputFilter .cgi
        </Directory>

        DirectoryIndex index.html objavi.cgi
        ErrorLog /var/log/apache2/objavi-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/objavi-access.log combined
        #ScriptLog /tmp/objavi-cgi.log

</VirtualHost>

A Virtual Host is a way of making Apache act like more than one web server.  We have one Virtual Host for Booki, and one for OBJAVI 2.  When you set up a Virtual Host you need some way for Apache to know which host is needed for a given request.  You can do this by giving your server more than one IP address, more than one DNS name, or in my case more than one port.  The Listen directive at the top says that we will be listening at port 8000 in addition to the normal HTTP port 80.  When a request comes in on port 8000 it will go to Booki and when it comes in on 80 it will go to OBJAVI 2.  Again, OBJAVI 2 must run on port 80.

For Booki we're using the mod_python plugin for Apache, so make sure it's installed.

Using OBJAVI 2

If you've done everything right you should be able to go to http://127.0.0.1:8000 and see Booki running, and go to http://127.0.0.1 and see OBJAVI 2 running.  OBJAVI 2 looks like this:

OBJAVI 2

OBJAVI is run from within Book from the Export tab when you're editing a book.  If all you want to do is create PDF's and EPUBs you may never need to look at this page.  There is one thing you can do from here that you can't do from Booki's Export tab, and that is to create output as Templated HTML.  To do that you choose Templated HTML as the Document Type.

Templated HTML is not a kind of e-book, but it is worthy of a brief mention.  One of the ways that Booki is different from Wikis like Media Wiki (used for Wikipedia) is that with a normal Wiki anyone can edit any document and the edit is available to the readers of the Wiki instantly.  A normal static website makes it easy to control who can update the content, but this control means that updating the content is more work.  What Booki introduces is the idea of generating a static website from a Wiki.  The Wiki is used by the book authors but is not seen by the book's audience.  When the authors have something ready to publish to the world they use OBJAVI to generate a static website and copy it to the public web server.

By default the HTML looks like the FLOSS Manuals website.  Because it is generated using templates, you can easily add your own stylesheets, corporate logos, and the like to make the generated site look the way you want it to.

You should definitely think about creating a templated HTML version of your book if the contents are likely to be updated frequently.  The website version of your book can then act as a supplement to the e-book version.  The stable content will be in the e-book and the latest minor tweaks and corrections will be on the website.