Lightweight High Performance Reference Database
The technique described in this lesson is useful for reference databases with low change frequency and large number of documents.I've created a Lightweight Directory on Notes.The technique used to build this database is useful for other reference databases with low change frequency and large number of records. Example: in the Netherlands a list of valid ZIP-codes could be stored this way. I've chosen the Directory on Notes because it best proves that the concept works.
The IBM Directory on Notes is not a successful application. This application has two major problems.
The first problem for the user is that the DoN does open very slow especially if the user opens the database for the first time. The administration of the unread marks causes this problem. For each database Notes keeps track of the (un)read documents even if in non of the views the unread marks are shown. IBM, Lotus and Iris have been working about this problem. One of the results of this effort can be seen by the user in the d_dir where next to the DoN the 'DoN Unread lists' database is listed.
The second problem is the replication of the database. The replication from an enterprise server to a workstations takes many (really many) hours. The initial replication setup takes 6 to 10 hours and the replication itself takes another reasonable amount of hours. During the Notes roll out the database was loaded using FTP and not by Notes replication. A result of this was that OS/2 staging servers with the DoN were not possible/allowed.
From 23 -25 February 1998 I was at the Lotus Technical Forum in Boston. One of the last presentations was about the new directory (old name and address book). A little demo was given. The directory contained about a 100K persons, but the database property "Documents" showed that the database contained 600 documents. At the end of the presentation was asked if we, as developers, could use this feature in our own applications. The speaker couldn't give or didn't know the answer.
Triggered by the directory of Notes release 5, I wondered if I could create a lightweight version of the Directory on Notes for Notes release 4. The answer is yes. Here are the characteristics of the IBM DoN and my lightweight version of it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Seeing the number of employees and number of documents in the database it is clear that information of multiple employees is stored in one document. The information of mulitiple employees is stored in one document as follows:
° a multi value field containing the employee names concatenated with the employee number to make each field value unique
° a field containing the links between the employee number and the field name in which the employee information is stored (the index)
° a field for each employee containing the employee information, the "fields" (name, phone, ...) within the field are seperated by one seperator character
° a field for the country code (only employees from the same country are stored in a document, this makes selective replication to workstation possible)
The following table gives an example of the contents of a document:
| Field |
|
| key |
Schroder, T (Taro)(788076112 Schroer, M (Marcel)(788076108 |
| empindex |
|
| I0 |
|
| I1 |
|
| I2 |
|
| Country |
|
| $anonymous |  |
The user can not open a document from this view as the user can in other Notes applications, because the opened document contains the data of multiple employees and in the application you can not determine which employee is selected in the view. So I created two forms, one for to do a lookup for employee information and one for selecting names to be inserted in the addressee fields in a memo. See the Lightweight Directory on Notes for the code to look up information.
The lesson learned here is that IBM can create a Directory on Notes which:
° can reasonable fast accessed (works over the phone for me),
° replicates easily over the enterprise servers
° can be replicated to a local workstation (Netherlands directory is 3.5Mb and replicates within 5 minutes)
Notes:
° The construction used is useful for other large reference databases as well
° The maintenance (insertions/deletions) of the data is more complicated then in other applications. You can no longer just delete a record by deleting a document in the database. Instead you have to process the index or rebuild the document will all the records within.
° IBMs strategical direction for the directory is DB2 and there is no intention to upgrade the DoN according this lesson learned
° Notes release 5 will deliver a new directory which can hold up to a million entries. At this moment it is not clear if this new directory can and will replace the DoN.
° For more information about lightweight reference databases see my subscription in the Best Practices Conference database (the document in this database describes the solution in more detail and contains the code to build a implified NAB this way). The Best Proactices database can be found on server D01DBR01 in the n-dir.
° For the Freelance presentation
about the lightweight see the Team.Connect website (link to Team.Connect from w3.coc.ibm.com, goto pizza sessies, select presentations, the Don presentation is "Notes Best Practices/Performance" of 16/04/1998
The April version of the Lightweight DoN is still available on the IBMNLP1 (9.132.37.141) in the 'test' directory.