The work to be done to revamp the Bibliomania site to full working order.
Text in green has not been budgeted for.
The Bibliomania site has been in mothballs for some two years, during which time only minimum system maintenance to keep the site online has been done to the live site. However the system code tree has been modified and kept current on a number of machines, particularly the development machines of TimP and JimW and on the Bibliomania development machine.
During this time the site has continued to grow in popularity, at a rate of some 100 new user registrations per day, the most popular feature would appear to be the bookmark functionality.
It is proposed to upgrade the site software and rebuild it so as that it continues to be supportable, to enable adverts to be easily inserted and to make the site a saleable asset.
Since the site was launched much has moved on, every element of the system software needs to be upgraded. We should consider upgrading the operating system and hardware as well.
|Product||Installed version||Current version||Site||Comments|
|Melati||0.55.3||0.7.0||http://melati.paneris.net/||Major, chargeable upgrade involving change to database structure|
|melatiShopping||alpha||1.0||http://shopping.paneris.net/||Minor, chargeable upgrade.|
|melatiBoards||alpha||1.0||http://boards.paneris.net/||Minor, chargeable upgrade.|
|webmacro||0.98p1||1.0||http://www.webmacro.org/||Free upgrade, requiring some work.|
|BerkleyDB||3.1.17||4.1||http://www.sleepycat.com/||Free upgrade, requires software modification|
|jakarta-oro||unused||2.0.5||http://jakarta.apache.org/||Free replacement, requires software modification|
|OROMatcher||1.1||1.1||http://jakarta.apache.org/||To be replaced entirely by jakarta-oro, requires software modification.|
|servlet||2.2||2.4||http://java.sun.com/||Free upgrade, should require little or no work|
Due to the upgrade to the full text index we need to rebuild it from scratch, this also allows us to reformat pages to insert banner adverts and make any proof reading changes we can.
This process is likely to take a considerable elapsed time. It has also, in the past, highlighted problems within the HTML sources which we should devote as much time as possible to correcting.
We should delete all user records and associated subscriptions and messages where no manual fields have been filled. These records were, on the whole, created when we had a policy of automatically creating a user record when we received an email, so the majority of the records are fictitious, created by spam.
We should add a creation date to the user records, retrofitting a date to existing records based upon the id field.
We need to re-establish the Secure Trading account for the shop to function.
It is intended that to be able to read a page you should be registered. This involves a challenge to non-logged-in surfers when they reach the page level of the site. This will not affect our Google footprint as this has been gained upon the meta data pages rather than the actual chapter pages.
Currently members sign up as a one step process. It is now common practice for registration to be a two step process where the user is sent a confirmation email which contains an URL they have to click on to complete the process. This ensures that the email entered is correct and that people are using their own email addresses.
We have been plagued by the NoMoreTransactions exception throughout the project. It appears to be brought about in two separate ways: firstly if a process dies due to a genuine exhaustion of available memory then this has caused the system not to free the transaction concerned and for all transactions after that to block; secondly there appears to be a problem on the increasingly enormous User table.
In the case where the system has genuinely run out of memory this can be rectified, it is believed, by detecting the error and automatically restarting the server (shutting down the JVM).
It is possible that running out of memory may be caused by errors in either Postgresql, its JDBC driver or Berkley DB all of which are to be upgraded, so the slow increase in memory usage may just go away.
In the case where there is an as yet unidentified problem with the User table there are a number of hopeful approaches:
So we can be sure that the NoMoreTransactions problem will be either solved or contained so that it is no longer the unsustainable system management burden that it has been.
Currently, when a user posts a message to a messageboard, if there is a single other user subscribed to the board who has a bad email address or a temporary fault with their account the bounce message is sent to the poster. This should be changed so that bounces are sent to the board manager as they are the person who can do something about it, such as unsubscribe, ban or delete that user.
Some MTAs lowercase email addresses, so we should either use a case insensitive search or lowercase all board names.
firstname.lastname@example.org and other bibliomania.com addresses are still handled by Maytech, they should be handled by the bibliomania machine. All aliases should be setup correctly and be documented.
We need to reduce the volume of logs produced. We need to analyse the errors that are produced. We need to process the access log on a different machine (dev machine).
We know from previous runs of the pagination process that there are a number of errors in the markup of the input data. These will be highlighted by the pagination process and should be corrected.
The study guides need to be checked against the list that David Pinching has and the missing ones need to be re-instated.
The Bibliomania system code contains no comments! This makes the learning curve for new maintainers nigh-on vertical. Comments should be added to the code wherever possible, at the least package descriptions and overviews are needed.
The installation, pagination, mail handling and logging procedures must be documented.