LogReport Foundation; Annual Report 2001
tools for computer/network log file analysis
1. Introduction
This annual report consists of the following sections:
- Introduction
- Activities in 2001
- Official Information
- Financial Statements
1.1 The goal of LogReport
LogReport aims to transform the data in log files of network/computer systems into useful information. Log files are usually treated like the waste of Information Technology activity. Files are sitting somewhere in a dark corner of a computer system and get only touched occasionally in case of reactive problem solving. This is unfortunate because log files contain the traces of computer activity, and by systematic analysis of this activity one can learn a lot of the behavior of the systems and the users. Stichting LogReport Foundation develops and maintains Open Source reporting software and documentation.
1.2 LogReport basic idea
Log files are distilled into an intermediate format (also known as DLF = Distilled Log Format) from which the reports can be generated. The intermediate step introduces a log file dependent and a report dependent part. The log file dependent part translates to the DLF format and the report dependent part composes the report from the DLF files. The intermediate format can be used as the store of a data warehouse which allows you to generate many reports from many log files from many sources.
2. Activities in 2001
2.1 Introduction
We report on the activities of the LogReport team during 2001.
The main task of this team is maintenance and development of Lire, LogReport's tool for performing an integrated analysis of all ones Internet and Intranet Services. Lire does this by automatically generating useful reports from raw logfiles from various services. Next to this work, the team has been doing promotional activities like writing articles, giving presentations and maintaining the website. Finally, the team has been administering the LogReport server.
LogReport's server, called hibou.logreport.org, is hosted at VIA Networks in Eindhoven, The Netherlands. The machine hosts the LogReport website http://logreport.org/, handles LogReport email, and hosts the 4 public mailinglists:<commit@logreport.org>
, <announcement@logreport.org>
, <questions@logreport.org>
, and <development@logreport.org>
. Furthermore, it hosts a CVS repository for version control of non-public documents and hibou's configuration data. The Lire code development is done using CVS on the SourceForge Open Source Development servers, http://sourceforge.net/projects/logreport/.
Various developers' meetings took place: On March 24th, there was a meeting at Jan Stap's place in Eindhoven, The Netherlands, attended by Joost, Jan and Egon. On May 8th, Joost, Egon and Wytze met at Joost's place in Eindhoven. On July 4th, Joost, Francis, Christoph and Egon met at the Libre Software Meeting in Bordeaux, France. All other developers' communication was done using email and IRC.
A more in-depth overview, specifically on Lire development, can be found in the variousJournal and Roadmapdocuments, which were presented at the board meetings in June, September and November 2001. A detailed list of the improvements on Lire can be found in the Lire NEWS file. Some highlights can be found on the LogReport history page on http://logreport.org/about/history.php too.
2.2 People
The year was started with Joost van Baal working for LogReport, 3 days a week. Furthermore, Anton Holleman, Peter Huisken, Gerrit Nijhuis and Jan Stap did work for the LogReport project on a volunteer basis. During the entire year, Teus Hagen, Wytze van der Raay and Jakob Schripsema formed the LogReport Foundation's board. Since only Joost could spent a large amount of time on LogReport work and Lire development, more contributors were welcome. In January, we started advertising vacancies. We've been talking to quite a few candidates.
In February 2001, Egon Willighagen from Nijmegen, The Netherlands joined the LogReport team. Especially his background in XML development made him suitable for Lire development.
On June 15, Francis J. Lacoste from Montréal, Canada and Christoph Lameter from Newark, California, USA, joined the LogReport team. Francis has a thorough background as a Perl hacker, Christoph knows how to set up a company; his TelemetryBox project, a Linux based distribution designed especially for network diagnostic purposes, could create some synergy with Lire.
On June 25, LogReport said goodbye to Anton, Peter en Gerrit as LogReport volunteers. Their fulltime jobs did not leave them sufficient time for LogReport work. Anton continues to be involved as external advisor to the board.
On August 23, Plamen Bozukov from Varna, Bulgaria, joined the LogReport team. He worked on, among other things, a Lire CGI interface.
On September 1, Christoph left LogReport: he started a fulltime job working on his TelemetryBox project.
On October 1, 2001 Joshua Koening from New York, USA and Wessel Dankers, from Bunnik, The Netherlands, joined the LogReport team. Wessel has been doing, among other things, PHP development. He is involved in several Linux development projects. Josh has a background as an internet and new media consultant. They both started their LogReport involvement by working on the LogReport website.
At that time, the LogReport team was up to its tasks. Hiring stopped.
Since November 1, Egon is no longer hired as an employee, but remains active as a volunteer. He decided to spent more time on his job as a bio-informatics researcher at Nijmegen University. He continues to do work for LogReport on an ad-hoc basis.
2.3 Lire Software Releases
In the year 2001, the following Lire versions have been released:
release date | version | filename(s) |
|
||
January 16 | 20010116 |
lr-20010116.tar.gz
|
February 19 | 20010219 |
lr-20010219.tar.gz
|
March 18 | Lire 20010318 |
lire-20010318.tar.gz
|
April 7 | Lire 20010407 |
lire-20010407.tar.gz
|
April 18 | Lire 20010418 | (pointrelease for Debian package) |
|
|
lire-20010418.tar.gz,
|
|
|
lire_20010418-1_all.deb
|
May 9 | Lire 20010509 |
lire-20010509.tar.gz,
|
|
|
lire_20010509-1_all.deb
|
June 26 | Lire 20010626 |
lire-20010626.tar.gz
|
June 29 | Lire 20010629 |
|
|
and 20010629.1 | (pointrelease) |
|
|
lire-20010629.1.tar.gz,
|
|
|
lire_20010629.1-1_all.deb
|
September 3 | Lire 20010903 |
lire-20010903.tar.gz
|
October 17 | Lire 20011017 |
lire-full-20011017.tar.gz,
|
|
|
lire-20011017.tar.gz
|
December 5 | Lire 20011205 |
lire-full-20011205.tar.gz,
|
|
|
lire-20011205.tar.gz
|
December 5 | Lire 20011205.1 | (pointrelease) |
|
|
lire-full-20011205.1.tar.gz,
|
|
|
lire-20011205.1.tar.gz,
|
|
|
lire_20011205.1-1_all.deb,
|
|
|
lire-20011205-2.noarch.rpm,
|
|
|
lire-html-20011205-2.noarch.rpm,
|
|
|
lire-pdf-20011205-2.noarch.rpm
|
Only since March 18 the LogReport software is called Lire. Pointreleases differ only in minor aspects from previous releases, these were made mainly to facilitate packaging. Note that since April 19 we ship Lire as a Debian package too, to facilitate installation on Debian GNU/Linux, and since December 16, we also ship Lire as a RPM, to facilitate installation on RPM-based system, like Red Hat Linux.
A Debian package based on Lire 20010626 was shipped with Christoph Lameters' TelemetryBox 1.2.4 in June 2001.
We list the main improvements in the various releases, as posted on the LogReport Announcement list. A more detailed overview is in the NEWS file, as distributed with Lire.
20010116 | The package no longer depends on GNU make. Lire is fully "autoconfiscated", which makes installation more flexible, and makes it easier to package Lire for various Unix distributions. Lire is ported to SUN Solaris. A "client" package is added, to be able to use Lire without installing the complete Lire suite. |
20010219 | Anlr_run script is added. This gives more userfriendly errormessage handling, and an easier interface for debugging. A local site config filedefaults.local is added: improved support for local configuration. Some more www reports are added. |
20010318 | Lire now generates XML output. The userfriendly configuration scriptlr_config is added. |
20010407 | Support for exim MTA log files is added. The internal format to represent reports is moved to pure XML, enabling more flexible output layer handling. |
20010509 | Reports can now get typeset in Adobe PDF format. Lire's documentation is converted to DocBook XML. |
20010629.1 | Reports can be typeset in HTML. PDF reports, as well as HTML reports, now can include images and graphs. |
20010903 | Various improvements and enhancements in the www reporting are made. Lire supports new output formats: LogML (an experimental XML application) and DocBook XML. |
20011017 | Various improvements and enhancements in the www and email reporting are made. Cisco ACL log files are now supported, as well as xferlog FTP log files. New supported output formats are XHTML and RTF. |
20011205 | CUPS (page_log) and LPRng (lp-acct) print log files are supported. W3C Extended Log Format webserver logs, as used by Microsoft's Internet Information Server is supported. Linux 2.2 ipchains firewall log, as well as Darren Reed's IP Filter log as shipped with *BSD's, are supported. New www, ftp and email reports are added. |
2.4 Statistics
We show some statistics, giving some indications on the work done by the LogReport team, as well as on the effectiveness of this work.
2.4.1 Downloads and installs
We do have figures about the number of downloads from our webserver. They're depicted in the next graph.
Number of Lire tarball downloads from
http://logreport.org/
in 4-week periods
Based on the Debian Popularity Contest, we estimate the number of people who've installed the Lire Debian package to about 200 at the end of 2001. (This was 0 until July: at that time there was no Debian Lire package yet.)
Please note that Lire is also available for download directly from Sourceforge. The Sourceforge-supplied statistics show that the number of downloads grew from 0 during January 2001 to 39 during April 2001 to 581 during October 2001.
2.4.2 Submits to the LogReport Online Responder
During the entire year, we've been running a so called Online Responder on <log@<service>.logreport.org>. People can submit their (anonymized and/or compressed) log files via email, and get a report back.
In July, we had about 4 different submitters, in August we had about 10 different submitters, in September about 40, in October about 60, in December about 50. At the end of the year, we had about 6 regular submitters, who submit their logs and keep coming back for a long time.
2.4.3 CVS Commits to the Lire code
There were between 100 and 700 commits on the Concurrent Version System holding the Lire code done monthly during 2001. The total number of commits was 5496. This number represents the number of changes to the Lire code.
2.4.4 Traffic on the LogReport mailing lists
During the year, the number of posts on our mailinglists has been increasing. However, we didn't succeed yet in building an impressively actively contributing user community. We did succeed in laying the seeds, though: some small contributions were made.
At the end of the year, about once a week, an external interested person posts to any of LogReport's mailing lists.
2.5 Cooperation with developers
2.5.1 External developers
A few people suggested improvements, using our mailinglists.
In July 2001, the LogReport team was contacted by John Punin, about the possibility of Lire interfacing with LOGML, an XML 1.0 application designed to describe log reports of web servers. Egon coded Lire support for this. Jesus M. Salvo Jr., the OpenJGraph maintainer, was contacted about tools to make the generated data easily accessible.
In August 2001, Egon contacted AWStats maintainer Laurent Destailleur about including his code in Lire. There were no real obstacles, since AWStats is free software. Synergy was achieved.
Joost Bekkers contributed valuable code to Lire, including convertors for proxy and cisco log files.
Since December 2001, Cédric Gross has been active on our mailinglists.
2.5.2 Licensing debate
The LogReport team has worked on improving external cooperation by clarifying the copyright ownership issue. A discussion took place among <people@logreport.org> (Joost van Baal, Plamen Bozukov, Francis Lacoste, Christoph Lameter, Jan Stap, and Egon Willighagen and <bod@logreport.org> (Teus Hagen, Jakob Schripsema and Wytze van der Raay) from mid July until the end of the year.
The outcome of this discussion has been layed down in the document titledCopyright and License of code, distributed by LogReport, as well as in the Licensing section of theFrequently Asked Question About Liredocument, as distributed with Lire. So, we're now able to respond in a professional way when someone wants to contribute code.
2.6 Website
On December 8, a totally revised LogReport website went life. Webmasters Joshua Koenig and Wessel Dankers did an excellent job here: the look and feel of the website is much more professional, while the backend is restructured into more maintainable code.
2.7 Promotion
On April 26, Joost did an improvised LogReport session at Moscow State University, which was attended by some Russian Computer Science students.
On November 30, Egon gave a presentation at Linux Kongress in Enschede, The Netherlands. An article on Lire is featured in the conference proceedings.
The September and November issues of LinuxFocus feature an article in two parts about Lire by Egon.
2.8 LogReport Future
On November 15 2001 Joost reported to the developers about the November Board Meeting. At that meeting, it was made clear once again, that LogReport funding is uncertain after the end of 2002. A long discussion has started since on how LogReport should proceed after 2002, and this discussion is continuing to date.
2.9 Contact information
web |
http://www.logreport.org/
|
|
|
<logreport@logreport.org>
|
|
|
|
irc |
#logreport on the OpenProjects.net IRC network |
|
|
sourceforge |
http://sourceforge.net/projects/logreport/
|
|
|
snailmail | Stichting LogReport Foundation |
|
p/a J.E. van Baal |
|
Strijpsestraat 48-A |
|
5616 GR Eindhoven |
|
The Netherlands |
3. Official Information
Stichting LogReport Foundation has been established on August 21, 2000 in Eindhoven, The Netherlands. The goal of the foundation is:
- to develop, maintain and distribute tools and knowledge for processing log files of network/computer system applications and for generating reports based on such log files;
- to stimulate the use of the tools and knowledge mentione above for the management of information systems;
- to stimulate authors of network/computer system applications to incorporate provisions in these applications for generating useful standardised and automatically processable information in log files;
- to contribute to the development and implementation of product-independent log file formats (standards);
- to create a forum for system administrators and software developers in the area of the application and analysis of log file information; and
- anything which is directly or partly related to the above, or can be beneficial to the above, in the widest sense.
3.1 Board
The board of Stichting LogReport Foundation consists of three members:
Teus Hagen | chairman |
Jakob Schripsema | secretary |
Wytze van der Raay | treasurer |
The composition of the board has not been changed since its start on August 21, 2000.
Four board meetings were held in 2001:
|
date | place |
|
April 12, 2001 | Oss |
|
June 13, 2001 | Oss |
|
September 12, 2001 | Oss |
|
November 14, 2001 | Oss |
Generally, board meetings have also been attended by Joost van Baal (liaison with LogReport operations) and Anton Holleman (advisor to the board).
3.2 Employees
The foundation has employed on average two part-time staff members during the year 2001:
Joost van Baal | November 2000 - now | software developer |
Egon Willighagen | February 2001 - October 2001 | software developer |
Wessel Dankers | October 2001 - now | webmaster |
In addition, a number of freelancers abroad have also performed paid remote development work for the foundation:
Christoph Lameter | USA | June 2001 - August 2001 |
Francis Lacoste | Canada | June 2001 - now |
Plamen Bozukov | Bulgaria | September 2001 - December 2001 |
Joshua Koenig | USA | October 2001 - now |
Finally, a number of volunteers have performed or are performing various tasks for the foundation.
3.3 Administration
The responsibility for day-to-day administration, handling of payments and other similar activities which was initially assigned to Anton Holleman, has been transferred in the first half of 2001 to Joost van Baal. The bookkeeping function has been taken care of by Wytze van der Raay, treasurer of the foundation. The salary administration and related work (interfacing to GAK, ARBO service etc) has been contracted out to De Wert Accountants in Eindhoven.
3.4 Fiscal year
The fiscal year of Stichting LogReport Foundation coincides with the calender year. Thus the second fiscal year of the foundation ran from January 1, 2001 until December 31, 2001. Therefore comparison with numbers from 2000, the first, incomplete, book year of the foundation (August 21, 2000 - December 31, 2000), does not always yield meaningful results, unfortunately.
3.5 Fiscal position
Based on its current activities, the foundation is not taxable for Dutch corporate tax ("vennootschapsbelasting") or value-added tax ("BTW").
On November 28, 2000, Stichting LogReport Foundation has been recognised by the Dutch fiscal authories as an organisation working for the general benefit ("algemeen nut beogende instelling") as meant in article 24, paragraph 4 of the Dutch Inheritance Act 1956.
4. Financial Statements
4.1 Balance Sheet per December, 31 2001 (after result allocation)
2001 |
|
2000 | |||
|
|
|
|
||
€ | € | € | € | ||
Fixed assets | |||||
Material fixed assets | |||||
Computer equipment | 1,853.63 | 2,717.55 | |||
Current assets | |||||
Accounts receivable | |||||
Interest to be received | 728.00 | 0.00 | |||
Costs paid in advance | 590.72 | 458.55 | |||
Cash | 54,080.27 | 2,261.89 | |||
|
|
||||
57,252.62 | 5,437.99 | ||||
|
|
||||
|
|||||
Own capital | 47,194.64 | 2,446.25 | |||
Short-term liabilities | |||||
Accounts payable | 4,574.85 | 303.32 | |||
Taxes and social charges | 3,600.18 |
|
2,094.75 | ||
Other liabilities | 1,882.95 | 593.67 | |||
|
|
||||
10,057.98 | 2,991.74 | ||||
|
|
||||
57,252.62 | 5.437.99 | ||||
|
|
4.2 Profit and Loss Account 2001
2001 |
|
2000 | |||
|
|
|
|
||
€ | € | € | € | ||
Other income | |||||
Donations received | 140,000.00 | 16,500.00 | |||
Other expense | |||||
Payroll expenses | 44,595.63 | 5,425.05 | |||
Depreciation of material | |||||
fixed assets | 1,402.92 | 220.34 | |||
Other operational expenses | 50,048.83 | 8,413.81 | |||
|
|
||||
96,047.38 | 14,059.20 | ||||
|
|
||||
|
|||||
43,952.62 | 2,440.80 | ||||
Interest earned | 795.77 | 5.45 | |||
Interest paid | 0.00 | 0.00 | |||
|
|
||||
795.77 | 5.45 | ||||
|
|
||||
Gross result from regular | |||||
operations before tax | 44,748.39 | 2,446.25 | |||
Tax | 0.00 | 0.00 | |||
|
|
||||
Net result | 44,748.39 | 2,446.25 | |||
|
|
4.3 General explanations
4.3.1 Basis for valuation and result determination
Assets and liabilities
Unless stated otherwise, assets and liabilities have been stated at their nominal values.
Material fixed assets
Material fixed assets have been stated at historical cost price, reduced with depreciation calculated linearly based on the estimated total useful life of the corresponding fixed asset.
4.3.2 Result determination
General
The following holds with respect to items included in the operational result: profits are only included if and for the part they have been realized in the reporting period, and losses and risks have been taken into account inasmuch they originate before the end of the reporting period.
4.4 Amplification of the Balance Sheet 2001
4.4.1 Material fixed assets
The material fixed assets can be specified as follows:
Inventory | Computer- | Total | |
equipment | |||
|
|
|
|
€ | € | € | |
January 1, 2001 | |||
Procurement costs | 0.00 | 2,937.89 | 2,937.89 |
Cumulative depreciation | 0.00 | (220.34) | (220.34) |
|
|
|
|
Book value | 0.00 | 2,717.55 | 2,717.55 |
|
|
|
|
Changes | |||
Desinvestments | 0.00 | 0.00 | 0.00 |
Investments | 0.00 | 539.00 | 539.00 |
Depreciation | 0.00 | (1,402.92) | (1,402.92) |
Depreciation desinvestments | 0.00 | 0.00 | 0.00 |
|
|
|
|
(863.92) | (863.92) | ||
|
|
|
|
December 31, 2001 | |||
|
|
|
|
Procurement costs | 0.00 | 3,476.89 | 3,476.89 |
Cumulative depreciation | 0.00 | (1,623.26) | (1,623.26) |
|
|
|
|
Book value | 0.00 | 1,853.63 | 1,853.63 |
|
|
|
|
Depreciation percentage | 20% | 50% | |
|
|
4.4.2 Accounts receivable
Accounts receivable consist of an estimate of the interest to be received on the savings account (due per May 1, 2002), an amount of expenses paid in advance for the months January and February 2002 with respect to a hosting contract with VIA NET.WORKS, an amount of expenses paid in advance for the period up to November 2002 with respect to ARBO service, and an amount of expenses paid to Gandi, Paris for domain name owner changes (to be effectuated in 2002).
4.4.3 Cash
Cash is kept on a business giro account and an associated savings account ("Kapitaalrekening") at Postbank N.V.
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Postbank giro account | 7,777.85 | 2,261.89 | |||
Postbank Kapitaalrekening | 46,302.42 | 0.00 | |||
|
|
||||
54,080.27 | 2,261.89 | ||||
|
|
4.4.4 Own capital
Reserve fund
The course is as follows:
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Value per January 1 | 2,446.25 | 0.00 | |||
Plus: net result | 44,748.39 | 2,446.25 | |||
|
|
||||
Value per December 31 | 47,194.64 | 2,446.25 | |||
|
|
4.4.5 Short-term liabilities
Taxes and social charges
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Salary tax | 3,046.08 | 1,233.56 | |||
Social charges bedrijfsvereniging | 554.10 | 861.19 | |||
|
|
||||
3,600.18 | 2,094.75 | ||||
|
|
Other liabilities
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Salary administration expenses | 128.52 | 181.29 | |||
Holiday allowances | 1,754.50 | 412.36 | |||
Savings salary | 0.01 | 0.02 | |||
Net salary | (0.08) | 0.00 | |||
|
|
||||
1,882.95 | 593.67 | ||||
|
|
4.4.6 Liabilities not shown in the balance sheet
The foundation has engaged in a year contract for internet hosting services with VIA NET.WORKS; the costs of this contract are currently € 2,851.20 per year.
4.5 Amplification of the Profit and Loss Account 2001
4.5.1 Income
Income over 2001 consists solely of four quarterly donations from Stichting NLnet with a total value of € 140,000.00.
4.5.2 Payroll expenses
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Staff salaries | 38,961.15 | 4,698.46 | |||
Social charges | 5,634.48 | 726.59 | |||
|
|
||||
44,595.63 | 5,425.05 | ||||
|
|
||||
|
|||||
Staff salaries | |||||
Salaries | 36,036.09 | 4,260.00 | |||
Salary tax on savings salary | 118.18 | 13.12 | |||
Holiday allowances | 2,806.88 | 340.80 | |||
Overhevelingstoeslag | 0.00 | 84.54 | |||
|
|
||||
38,961.15 | 4,698.46 | ||||
|
|
||||
|
|||||
Social charges | |||||
Social charges | 5,550.90 | 726.59 | |||
ARBO service | 83.58 | 0.00 | |||
|
|
||||
5,634.48 | 726.59 | ||||
|
|
4.5.3 Other operational expenses
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Other staff expenses | 42,331.95 | 376.14 | |||
Office expenses | 5,683.85 | 1,175.50 | |||
General expenses | 2,033.03 | 6,862.17 | |||
|
|
||||
50,048.83 | 8,413.81 | ||||
|
|
||||
|
|||||
Other staff expenses | |||||
Travel expenses staff | 1,740.73 | 74.86 | |||
Travel expenses volunteers | 225.22 | 301.28 | |||
Compensation volunteers | 76.00 | 0.00 | |||
Cost of freelancers | 40,290.00 | 0.00 | |||
|
|
||||
42,331.95 | 376.14 | ||||
|
|
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Office expenses | |||||
Housing expenses | 413.77 | 0.00 | |||
Office materials | 38.62 | 44.92 | |||
Forwarding charges | 16.51 | 123.88 | |||
Telephone costs | 1,038.22 | 0.00 | |||
Internet expenses | 3,716.43 | 995.38 | |||
Computer expenses | 460.30 | 11.32 | |||
|
|
||||
5,683.85 | 1,175.50 | ||||
|
|
||||
|
|||||
General expenses | |||||
Professional books | 321.63 | 0.00 | |||
Subscriptions & contributions | 258.56 | 35.62 | |||
Conferences/courses | 90.76 | 192.86 | |||
Publicity | 22.94 | 0.00 | |||
Accountancy fees | 506.52 | 181.29 | |||
Consulting fees | 0.00 | 6,273.02 | |||
Travel expenses board | 345.51 | 172.58 | |||
Other board expenses | 279.52 | 0.00 | |||
Representation costs | 22.23 | 0.00 | |||
Bank expenses | 185.50 | 6.80 | |||
Calculation differences | (0.02) | 0.00 | |||
Euro-calculation differences | (0.12) | 0.00 | |||
|
|
||||
2,033.03 | 6,862.17 | ||||
|
|
4.5.4 Interest earned
2001 | 2000 | ||||
|
|
||||
€ | € | ||||
Credit interest Postbank giro account | 67.77 | 5.45 | |||
Credit interest Postbank Kapitaalrekening | 728.00 | 0.00 | |||
|
|
||||
795.77 | 5.45 | ||||
|
|
4.6 Allocation of net result 2001
In anticipation of a board decision to be taken, the net result over 2001 ad € 44,748.39 has been added to the reserve fund.