Database::BI business intelligence for data files

Perl · Mojolicious · no database required

Database::BI - Web-based Business Intelligence viewer for data files

Database::BI is a self-contained web application that reads the data files you already have — CSV, PSV, TSV, SQLite, JSON, XML and XLSX — and presents them as sortable tables, nine kinds of D3 chart, maps, word clouds, pivots and saved canvases. There is no database to load and no import step: each request reads the file as it is now.

What you get

Everything below is served from the same application, against the same data pipeline, so a filter you add on a table is the filter the chart behind it draws from.

Tables

Open anything, anywhere

The home page scans a data directory; /browse walks the whole filesystem and opens any supported file, including a table inside a multi-table SQLite database, a file on another host over SFTP, or a table scraped from a public web page. Files can also be dropped onto the page.

Tables

Sort, reorder, filter

Click a header to sort, drag headers to reorder — both remembered by column name in the browser. The quick-filter bar builds server-side conditions with eq, ne, contains, starts, lt, le, gt, ge, empty and notempty.

Shape

Join and combine

A left join appends columns from another table on a shared key, and joins chain. Combine stacks one file under another on a unified column set, leaving blanks where a source has no such column.

Charts

Nine D3 chart types

Line, bar, pie, scatter/bubble, heatmap, stacked bar, combo bar and line, treemap and histogram. Each page exports SVG, PNG and PDF, takes a relative time range, and accepts calculated fields. See the chart pages.

Charts

Maps and word clouds

Plot latitude and longitude columns, or geocode a place column locally, with clustered markers, heat layers and colour rules. A word cloud counts the words of a text column, one vote per row.

Analysis

Pivot and statistics

A crosstab with row and column dimensions and a shared measure vocabulary (sum, avg, median, stdev, p<N> and more), plus per-column statistics computed the same way.

Analysis

Calculated fields and groups

Virtual columns from an expression, named value groups that map many raw values onto one label, date patterns that normalise a column to ISO, and cross-row columns such as running totals, ranks and z-scores.

Canvases

Dashboards you can share

Save a configuration as a canvas of cards on a 12-column grid, arrange them by dragging or with the keyboard, add titles and explanations of your own, and let readers cross-filter and drill through. Canvases have bookmarks with undo, parameters, share links and an embed route. How canvases work.

Data out

Export what you see

The current logical view — after joins, filters and deduplication — goes out as CSV, SQLite, JSON or XLSX, and every chart saves as SVG, PNG or a one-page PDF.

Preferences

Light or dark

Every page, table and chart comes in a light and a dark theme. Each reader chooses on the Preferences page, or follows the device. How preferences work.

How it fits together

  1. A file is a table. A small adapter reads CSV, SQLite, JSON, XML or XLSX through Database::Abstraction and hands back rows and columns. Nothing is written back to the file.
  2. One pipeline, many views. Joins, combines, date normalisation, calculated fields, named groups, filters, relative ranges and cross-row passes happen in a fixed order, so a table, a chart, a pivot and a canvas card over the same parameters see the same rows.
  3. Charts are snippets, not images. Each chart page renders a D3 script into the page, which is why a chart can be resized, exported as a file and cross-filter another card on a canvas.
  4. A canvas is a saved configuration. The document holds the datasets, the joins, the visuals and their grid positions; opening it re-renders every card against today's data.

Quick start

cpanm Database::BI          # or: cpanm --installdeps .  in a checkout
script/database-bi          # development server, reloads on edit
script/database-bi data/store.db   # the same, showing that one database
hypnotoad script/database-bi   # preforking server for real use

Then put your data files in data/ beside the launcher, or point data_dir at a directory of your own, and open the home page. Naming one database shows it there instead. The application ships a tutorial in plain English at /tutorial; it assumes no Perl and is the place to start.

Your rows stay put. The application reads files from the machine it runs on and keeps no database of its own. The optional AI features are off unless a key is configured, and even then prompts carry column names and types or aggregate text — never row data or file paths. Map pages never send what you type anywhere: there is no place-search box.

Preferences

The Preferences link at the top of every page opens /preferences, where a reader chooses how the application looks to them. It has one setting so far, the theme:

Same as this device
The default. Light or dark, following the setting on the computer or phone, even if that changes while a page is open.
Light
Dark text on a pale background.
Dark
Pale text on a dark background, for the tables and the charts as well as the pages around them. Map tiles keep their usual colours.

A choice takes effect at once, with no Save button, and pages open in other tabs follow it. Saved charts and printed pages always use the light colours, so they look the same to whoever receives them, and so does a canvas embedded in another site, whose colours it cannot know.

Preferences belong to the browser. They are kept in its localStorage and never reach the server, so there is nothing to configure and no account to sign in to, and another browser or computer keeps its own. If the browser refuses to store a choice, the page says so and the choice lasts only until the next page.

Where to go next