
Flatstore Module

Jan Janak

   FhG FOKUS

   Copyright  2004, 2005 FhG FOKUS
     _________________________________________________________

   Table of Contents
   1. User's Guide

        1.1. Rotating Log Files
        1.2. Dependencies
        1.3. Exported Parameters

              1.3.1. flush (integer)

        1.4. Exported Functions
     _________________________________________________________

Chapter 1. User's Guide

   Flatstore is one of so-called SER database modules. It does
   not export any functions executable from the configuration
   scripts, but it exports a subset of functions from the
   database API and thus other module can use it instead of, for
   example, mysql module.

   The module does not export all functions of the database API,
   it supports only one function, insert. The module is limited
   but very fast. It is especially suitable for storing
   accounting information on sites with extremely high traffic.
   If MySQL is too slow or if you get a huge amount of accounting
   data then you can consider using this module. Note that the
   acc module is the only module that was tested with flastore.

   The format of the files produced by this module is plain text.
   Each line consists of several fields, fields are separated by
   | character. New information is always appended at the end of
   the file, searching, deleting and updating of existing data is
   not supported by the module.

   The acc module can be configured to use flatstore module as
   database backend using the db_url_parameter:
modparam("acc", "db_url", "flatstore:/var/log/acc")

   This configuration options tells acc module that it should use
   the flatstore module and the flatstore module should create
   all files in /var/log/acc directory. The directory must exist
   and SER processes must have permissions to create files in
   that directory.

   Name of files in that directory will follow the following
   pattern:
<table_name>_<process_name>.log

   For example, entries writen by SER process 8 into acc table
   would be written in file acc_8.log. For each table there will
   be several files, one file for every SER process that wrote
   some data into that table. The main reason why there are
   several files for each table is that it is much faster to have
   one file per process, because it does not require any locking
   and thus SER processes will not block each other. To get the
   complete data for a table you can simply concatenate the
   contents of files with the same table name but different
   process id.
     _________________________________________________________

1.1. Rotating Log Files

   There is a new SER FIFO interface command called flat_rotate.
   When SER receives the command then it will close and reopen
   all files used by flatstore module. The rotation itself has to
   be done by another application (such as logrotate). Follow
   these steps to rotate files generated by flatstore module:

     * Rename the files that you want to rotate:
cd /var/log/acc
mv acc_1.log acc_1.log.20050605
mv acc_2.log acc_2.log.20050605
mv acc_4.log acc_3.log.20050605
...
       Note that at this point SER will still be writing all data
       into the renamed files.
     * Send SER the fifo command to close and reopen the renamed
       files:
serctl fifo flat_rotate
       This will force SER to close the renamed files and open
       new ones with original names, such as acc_1.log. New files
       will be open at the point when SER has some data to write.
       It is normal that the files will be not created
       immediately if there is no traffic on the proxy server.
     * Move the renamed files somewhere else and process them.
     _________________________________________________________

1.2. Dependencies

   None.
     _________________________________________________________

1.3. Exported Parameters

1.3.1. flush (integer)

   Enable or disable flushing after each write.

   Default value is 1.
     _________________________________________________________

1.4. Exported Functions

   This module does not export any functions usable from the
   configuration script.
