TDDI48 Project, Part 4: Message Forum

Written by Jonas Kvarnström. Please report any bugs, inconsistencies or vague requirements as soon as possible. Please include the course code "TDDI48" in the subject!

Revision history since 2007-02-06

Introduction

In this part of the project, you will build on what you have learned in the first part in order to create a message forum. We have already shown a number of examples of how a message forum could look and what functionality can be present.

We would like to recommend the following basic structure:

  1. Somewhere in your system, you can access a list of all message forums (implemented as a servlet or a JSP page). Here you can see general information about each forum (number of topics, total number of posts, maybe the date of the latest post, and so on).
  2. Clicking on a forum link takes you to the list of topics (subjects) within that forum. This list is split into multiple pages, since there may be many topics in your forum and an user should not have to view thousands of topics in a single page.
  3. Click on a specific topic to view the messages within that topic.

Throughout the project, please remember once again that the number of exercises or extensions you implement is not the only criterion on which your grades are based. Do not neglect basic usability or code quality in order to save time to get additional extensions implemented. Remember, if your project is lacking in usability, or your code is not of sufficiently high quality, your project might not be accepted or you may not achieve the grade you are aiming for!

Since you already have some experience developing web applications, these instructions will not be quite as detailed as in part 1. Please use common sense, and ask us if anything is unclear!

Exercise 1: Message database

In this exercise you will create a few SQL tables to store forum, topic and message information. In this lab description, the tables will be called "forums", "topics" and "messages".

Notes on Unique IDs

You will have to generate unique IDs for messages, topics and forums, even though people may be adding such new items concurrently. The best way of doing this in MySQL is to make the ID column an auto-increment column (read more about this in the MySQL manual). When you inserting a new row, the auto-increment column will automatically be given a new unique value.

You can then use the LAST_INSERT_ID() SQL function to determine which ID was just generated, which may be necessary for example when a new topic and a corresponding message is to be created at the same time. SELECT LAST_INSERT_ID() returns the last auto-incremented ID that was generated in the same Connection – if other IDs are generated by other database connections they will not interfere, but beware of the possibilities of race conditions if you reuse the same Connection for multiple insert queries that may be performed concurrently.

In JSTL, this may mean you have to place the insert query and the select query within a transaction tag, in order to ensure that the same Connection is used for both queries.

Forum Table

Create a forum table containing information about each forum. A forum should keep track of its forum ID (a unique integer identifying the forum) and its name. If you eventually want to store additional information about each forum in the forum list, now would be a good time to think about this and add the fields right now to save some time later! For example, you may want to provide an extended description of the forum. Take a look at the MozillaZine forum list and the IntelliJ forum list for inspiration.

Note that some of the types of information you might want to show in a list of forums can be implicitly available through links to other tables. For example, the number of unique topics in the forum can be calculated through a join with the topic table discussed below (grouping by the forum ID and calculating the COUNT of the number of topics), the date of the latest post to the forum can be calculated through a join with the message table (calculating the MAX of the post dates for all messages in each forum), and so on.

Storing a copy of this information (the number of topics and the latest post date) in the forum table leads to a non-normalized database, with duplicated information; one must be very careful to use transactions properly, to make sure that tables are kept synchronized. For example, no topic must ever be added or deleted without the forum table being updated appropriately, or the number-of-topics field of the forum table will not show the correct value. On the other hand, information about each topic can be found extremely quickly, without the need for table joins. With a normalized table (no duplicated information), keeping data in sync is a smaller problem, but the queries used to extract information are somewhat more complex. Whether the use of table joins to calculate data is a problem in practice depends on the size of the forum, the optimizations made by the database engine, and the indexes that you create. For this project, the choice is up to you – what is important is that you are aware of this choice and its consequences.

Topic Table

Create a topic table containing information each topic within a forum. A topic should keep track of its ID (a unique integer identifying the topic), its name (which will be displayed in the list of topics), and the ID of the forum it belongs to. Some examples of additional information you could add (some of which leads to a non-normalized database): The number of messages in the topic, the date of the latest post to the topic, and the number of times the topic has been viewed.

Message Table

Create a message table that stores all information about a single message. The message should at least contain fields for a message ID (a unique integer identifying the message), a user ID identifying the user who wrote the message (this should be the key in the user database, in other words the e-mail address), the date and time when the message was written, a subject line, and the actual message – and the ID of the topic it belongs to.

Exercise 2: Adding and viewing forums

In this exercise you will create a user interface for adding new forums.

Forum List

Create a servlet or JSP document that displays a list of all forums. See MozillaZine Forums for an example of how a list of forums could look. So far this will only display an empty list, since you have not yet created any forums!

The user should probably not have to be logged in to list the available forums. The forum should be available to anyone. An account is only needed to make changes (for example, to post new messages).

The forums in the list do not have to be "clickable" yet (this will be added in the next exercise).

Creating New Forums

Create a web-based interface (using HTML forms, servlets and/or JSP documents) to make it possible to add new forums.

The user has to be logged in to do this. If the user tries to add a new forum without being logged in, he should be forwarded to a login page. If he logs in successfully he should once again be forwarded back to "create new forum".

Later you may restrict the creation of new forums to users who are administrators, but for now it's OK if you allow everyone to create new forums.

Add Example Forums

Use the interface to add a number of new forums. In order to make sure you can test the forum list properly, please add at least a dozen forums, even if most of them are just named "forum n".

Exercise 3: Adding and viewing topics and messages

In this exercise you will create a web-based user interface where you can add new topics and messages and view existing topics and messages.

Viewing the Topic List

Create a servlet or JSP document that displays a list of all topics within a given forum. Modify the forum list so that each forum name is a link to the topic list servlet or JSP document. The forum ID can be specified as a query parameter (see the example below).

For scalability reasons, there should be a limit to the number of topics that can be shown in a single page (let's say 4, while we are testing – 30 may be more suitable for production use). If there are more topics, then only show the first 4 topics and provide links to other topic pages. In these links, you should use query parameters to specify the start index ("/forum/viewtopics?forum=12&start=8").

The number of topics shown on each page should not be hardcoded. Instead, implement it as a servlet or webapp initialization parameter to be configured in web.xml, so that it can be tuned and changed easily by the web site administrator without having access to the source code. This may be a good idea for some other parameters too!

Which other topic pages should you link to? You should always have a separate link that points to the next page (if one exists) and to the previous page (if one exists). If you don't have more than 300 topics per forum (for example), and you show 30 topics per page, then there is a maximum of 10 different topic pages. Then you might want to show links to every page – something like this:

<-- Previous   [1]  [2]  --3--  [4]  [5] -->
    

But some larger forum systems on the web may contain hundreds of thousands of messages in a single forum. Let's suppose that there are 300,000 topics in one forum, and that you show 30 topics per page. That's still 10,000 pages! If you provide links to every single page, then there will be entire screens full of numbered links... This can be solved by only showing links to a few pages before or after this page (58 59 60 *61* 62 63 64, which makes it a bit difficult to "travel long distances"), by decreasing the density of links as you get further from the current page (1 2000 4000 4900 4990 4998 4999 *5000* 5001 5002 5010 5100 6000 8000 12000), by using a drop-down box, or by some other method that you invent.

Trusting User Input

Note that the parameters given to the forum list can be affected by the user. What happens if you manually change the link to these values?

Adding New Topics

On every page of the topic list, there should be a "new topic" link or button. Selecting "new topic" should take you to a web-based interface (using HTML forms, servlets and/or JSP documents) where you can add a new topic. When you add a new topic, you should enter a subject line as well as a message. This should create a new topic belonging to the current forum, and a new message belonging to the new topic.

The servlet or JSP should send back a message stating that the message was successfully added to the message forum (if it was successfully added!), and the message should include a link back to the message forum.

Viewing and Modifying a Single Topic

You should also be able to click an existing topic (subject). This should take you to a paged list of all messages in that topic, sorted in some suitable order such as inverse date order. The list should contain the actual messages, not just a list of message subjects. (By "paged" we mean that there should be a limit to the number of messages shown in a single page, just like in the topic list.)

The list should have a button for creating a new message within the topic. This functionality ís quite similar to the way you create a new topic (need to specify a subject, a message, and possibly some more information), but instead of creating a new topic the message should be added to the existing topic.

SAFE HTML: As in the user account database, the message display code must convert any characters that are "unsafe" into HTML entity codes. This should be done when the message is being displayed, so that the original message is available unaltered in the database. The database should be independent of the use of HTML! In JSTL, you may be able to use a single tag for this conversion... Do you remember which one?

PARAGRAPHS: When you display messages, you should convert double newlines into an HTML paragraph mark, <p>. This may require calling a Java utility method that you write.

Statistics

If you added (possibly non-normalized) fields for statistics, make sure they are updated correctly. For example, if you added a field for the number of times a topic has been viewed, then it should be updated (and stored in the SQL database) each time you display the list of messages in a topic.

If you need to update more than one table, you should use transactions, so that bugs or hardware cannot cause a crash between updating one table and updating the next, leaving the database in an inconsistent state!

Exercise 4: Using the forum

Please test your forum. Add a few new forums. Add a number of topics and add messages to them – at least a few dozen of them, so that you can test the message display code. Feel free to copy and paste from web pages or to write a small program that generates random words to place in the database, as long as you have a reasonable amount of messages! (Reasonable means at least a few dozen!)

Exercise 5: Checkpoint

The application you are writing is quite large, and from time to time you need to do a feature freeze, where you take a break from implementing new features and concentrate on fixing bugs and smoothing out rough edges.

Do not ignore this checkpoint! Even if you feel like you have enough to do anyway, the checkpoint is important for the final quality of your code, which in turn is important for your grade. This exercise should take at least several hours, possibly much more!

As a first step, take a look at the application yourselves. Imagine that someone else has written this message forum, and that you want to use it on your own web site (or you are a casual user visiting a web page where the forum is used). Is there anything you would complain about? Anything you think could be done in a better way? Are error messages difficult to understand? Is the application difficult to use? Are some parts of the application "disconnected" from other parts, making it difficult to navigate between different functions? Is it ugly? If so, take some time to polish the design and the user interface.

Then, take a look at your code. Check your error handling, fix your comments, ensure that you don't have a lot of old obsolete code lurking in your .java or .jsp files, and in general clean up the classes, interfaces and JSP pages that you use. Check whether any refactoring is needed — maybe there's a place where you're throwing a plain IOException although it would be much better if you created a new exception type instead, maybe your header or menu definition is duplicated in many JSP documents, maybe some classes or methods or documents should be renamed, and maybe you really ought to introduce a couple of separate classes or JSP pages handling certain things instead of cramming too many things into a single class or JSP page. You should be proud of the quality of the code you have written!

Finally, when you have done this, you should ask your lab assistant to take a look at your system and possibly your code.


Jonas Kvarnström

[Check links]