Development:UserDash
UserDash is the extension that captures most of the websites unique functionality, that MediaWiki does not provide. The source code for this extension can be found in Gitlab.
PHP/database (server-side) component
The UserDash extension uses PHP mainly for the use of additional database tables to handle user-specific data, and to add widgets to most pages when the user is logged in.
The special pages Dashboard, Profile and Contents are also provided through the extension.
All PHP classes are in the namespace MediaWiki\Extension\UserDash.
Database
UserDash makes use of 3 extra database tables to provide the functionality required. None of the table enforce any foreign key constraints yet; this may be something to keep in mind for the future.
Ratings table
Ratings are stored in the table userdash_vote, defined as follows:
CREATE TABLE IF NOT EXISTS /*_*/userdash_vote ( `ud_rating` int(5) NOT NULL, `ud_date` datetime NOT NULL, `ud_user` varchar(255) NOT NULL, `ud_page_name` varchar(255) NOT NULL, PRIMARY KEY (`ud_page_name`,`ud_user`, `ud_date`) ) /*$wgDBTableOptions*/;
- ud_rating stores the rating as an integer.
- ud_date stores the date of the rating.
- ud_user stores the username.
- ud_page_name stores the name of the page being rated on.
Note that ud_date is also included in the primary key. This is to facilitate recording the progress of a user through the change in their votes over time. Also, to prevent an indecisive user from creating many votes in one hour (which would not be very reflective of their progress), Rating.php updates the latest vote instead of inserting a new one if a new vote is made within one hour of the previous vote. This is not enforced by the database, and things that read from the vote table don't - and shouldn't ever need to - make the assumption that two votes by a user won't be within an hour.
Visits table
Pageviews are tracked in the table userdash_visit, defined as follows:
CREATE TABLE IF NOT EXISTS /*_*/userdash_visit ( `ud_date` datetime NOT NULL, `ud_user` varchar(255) NOT NULL, `ud_page_name` varchar(255) NOT NULL, PRIMARY KEY (`ud_page_name`,`ud_user`,`ud_date`) ) /*$wgDBTableOptions*/;
This table simply records every page view made by each logged-in user. Nothing uses it at the moment.
Tags table
Page tags, such as bookmarks and TA requests, are stored in the table userdash_tags, defined as follows:
CREATE TABLE IF NOT EXISTS /*_*/userdash_tags ( `ud_date` datetime NOT NULL, `ud_user` varchar(255) NOT NULL, `ud_page_name` varchar(255) NOT NULL, `ud_tag` varchar(20) NOT NULL, PRIMARY KEY (`ud_page_name`,`ud_user`, `ud_tag`) ) /*$wgDBTableOptions*/;
This table is somewhat similar to the ratings table, except rather than storing an integer rating, it stores a tag name of 20 characters. Note that ud_date is not in the set of primary keys; tags are removed with a DELETE, and added with an INSERT.
Helper classes
Access to the database is (mostly) handled by methods in "helper classes".
Rating
The Rating class is in Rating.php, and contains methods that interface with the userdash_vote table. More importantly, it also contains the canPageBeRated method, which returns whether a page can be rated, i.e. displays the TagBar and the Rating widget.
Tags
The Tags class is in Tags.php, and contains methods that interface with the table userdash_tags. Check the file itself for more info.
Visits
This class does not exist. All database access regarding visits are handled where they are needed.
Hooks
(Most) hooks are implemented in Hooks.php, including:
| Hook name | Comments |
|---|---|
| onArticleViewHeader | Renders the tag bar at the top of every page that can be rated, if the user is logged in. |
| onArticleViewFooter | Renders the rating widget at the bottom of every page that can be rated, if the user is logged in. |
| onBeforePageDisplay | Loads Javascript modules where necessary, and sets Javascript config variables. |
| onLoadExtensionSchemaUpdates | Creates the extension tables if they do not exist, when maintenance/update.php is run. |
| onSpecialMovepageAfterMove | Propagates changes in page name to the extension tables when a page is moved. |
| onArticleDeleteComplete | Removes all data related to an article when an article is deleted. |
| onSkinBuildSidebar | Adds Dashboard link to the top navigation bar if, and only if, the user is logged in. |
PageGraph
PageGraph.php contains the only hook outside of Hooks.php, onParserFirstCallInit, which makes the wiki render the pagegraph when it sees the tag <pagegraph/>.
API interfaces
A simple API interface is also offered, mainly for the javascript widgets to get the data they need through API requests. For all of these APIs, the user is obtained automatically from the logged-in user.
ApiRating
ApiRating handles the querying and setting of ratings. Ratings may be queried per page, in which case the count of all ratings in that page will be returned, as well as the user's rating for that page; or ratings may be queried per user, in which case all ratings a user has made will be returned. The parameter recent may also be added to only get the last time a user's made a new rating, which is used for determining when to update the page map.
ApiTags
ApiTags handles the querying and setting of tags. As with ratings, they may be queried per page or per user, though when querying per page, only the tags the current user have set on the page will be returned.
ApiDataManagement
ApiDatamanagement handles tasks that concern more than a single component, such as querying for all update times and the tags/rating on a page (which is a batch of all extra, optional data required for the typical page) or deleting all data belonging to a user.
Dashboard, Contents and Profile
Dashboard.php. Contents.php and Profile.php generates Special:Dashboard, Special:Contents and Special:Profile respectively. They generate the HTML that forms the page, and include Javascript modules specific to the the page they generate.
Javascript (client-side) component
Compared to the old MESH, the client handles more of this website's data, with the entire page map being cached client-side. Certain page elements, such as the page graph, read from the page map and perform filtering/sorting locally, without needing to make more requests to the server.
The javascript component is composed of various modules (defined in extension.json for the php part of the extension), which, in turn, contains a number of javascript files/objects:
| Module name | Object name | File name | Comments |
|---|---|---|---|
| ext.userDash.data | mw.userDash.dataRepository | ext.userDash/dataRepository.js | Handles the client-side data cache, and hands data out to code that needs it. |
| ext.userDash.rating | mw.userDash.rating | ext.userDash/rating.js | Handles the rating system. |
| ext.userDash.tagBar | mw.userDash.tagBar | ext.userDash/tagBar.js | Handles interaction with the top bar, which allows the user to set Bookmarks, among other things. |
| ext.userDash.dashboard | mw.userDash.ratingHistogram | ext.userDash/ratingHistogram.js | Handles the rating bar chart, which can be clicked on to reveal articles with a certain rating. |
| mw.userDash.ratingChart | ext.userDash/ratingHistogram.js | Handles the rating chart, a graph of a user's ratings over time. | |
| mw.userDash.donutProgress | ext.userDash/ratingHistogram.js | Handles the donutProgressChart, a simple circular chart displaying the user's progress. | |
| ext.userDash.graph | mw.userDash.graph | ext.userDash/graph.js | Draws a page graph based on data provided. |
| N/A | ext.userDash/PriorityQueue.js | A library downloaded from npm, included as a minified js file. | |
| ext.userDash.localMap | mw.userDash.localMap | ext.userDash/localMap.js | Provides appropriate parameters to graph.js based on page elements. |
| ext.userDash.contents | mw.userDash.contents | ext.userDash/contents.js | Handles the contents page, in general. |
| mw.userDash.contentsTable | ext.userDash/contentsTable.js | Handles the table part of the contents page. | |
| mw.userDash.globalMap | ext.userDash/globalMap.js | Handles the page map part of the contents page. | |
| ext.d3 | N/A | ext.d3/d3.min.js | External library D3, draws graphs of all kinds. |
All javascript files are in {EXTENSION ROOT}/resources/*.
dataRepository
mw.userDash.dataRepository is a javascript object that handles access to the pageMap. Javascript code may call mw.userDash.dataRepository.register(callback, param) to request a call to callback(pageMap, param, pageNameMap) once the pageMap is confirmed to be up to date. This provides the page map, a map from page names to page map indices, and a single parameter object to the callback function.
The local storage variables pageLastUpdate, ratingLastUpdate and visitLastUpdate keep track of when the local copies of page data, rating data and page-visit data were last updated. Pages are updated the least frequently (most likely barely at all once the wiki goes into production), followed by ratings, followed by page views. Separating the update for these three components allows the client to only fetch the data it needs.
The last update times are then fetched from the server through api.php?action=userdash&format=json&pagetitle={THE PAGE TITLE HERE}, which also registers the page view, all in one request.
If the data on the server is more updated than the local cache, then the data will be fetched from the server (a bit more than 220KB worth)
The dataRepository allows crucial data to be cached by the client, so that the client doesn't have to request the same/similar data from the server for each page load.
pageMap
The pageMap, stored in the browser's local storage (window.localStorage) as a string and in the dataRepository (mw.userDash.dataRepository) as a variable, is a local cache of all metadata related to pages. It is a javascript object structured as thus:
- (Object)
- nodes[] (Array of pages in the wiki)
- category[] (Array of categories a page belongs to)
- dependantRelations[] (Array of names of articles depending on a page)
- dependencyRelations[] (Array of names of articles a page depends on)
- group (Namespace ID; currently doesn't serve much of a purpose)
- id (Page name; the naming is inconsistent with the wiki)
- priority (Priority of the article)
- rating (The rating applied by the user, or -1 if there is no rating)
- satisfied (Whether all of the article's dependencies have been satisfied, i.e. rated 3 or above)
- summary (Article summary, for tooltip and sidebar in the pagegraph)
- links[] (Array of dependency links in the wiki)
- source (The name of the page depending on another page)
- target (The name of the page being depended on)
- value (The strength of the dependency)
- nodes[] (Array of pages in the wiki)
The pageMap holds almost all information required for most functionality, and it holds redundant information in order to speed up execution; dependantRelations and dependencyRelations hold information that is entirely captured by the array links[], and the properties category and priority in an element in links[] can also be obtained from the nodes[] array. This information is duplicated so that filtering is easier to run and implement.
In general, the pageMap is designed around the page graph; the pageMap object, when passed to mw.userDash.graph.draw, yields a complete page graph with all pages, with no additional processing needed. This is because most pages include a page graph, which means that it would be beneficial to minimize processing for the page graph.
In addition, the pageNameMap (window.localStorage.getItem("pageNameMap") or mw.userDash.dataRepository.pageNameMap) is a map from page names to their index in pageMap.nodes, and reduces the common operation of finding a page with a certain name from linear time, down to the time taken to address a javascript map (which should hopefully be shorter).
Ratings
Ratings represent how much a student is familiar with an article. They go from 1-5 (Which can be configured in the extension settings, but not many things implement the ability to handle different ratings), with a value of 0 representing a reminder to rate on an article.
Ratings are handled by mw.userDash.rating in rating.js.
When a rating is made, mw.userDash.rating.submitRating is called, which sends the rating request to the wiki. Additionally, the pageMap is updated directly, along with the ratingLastUpdateTime. This can lead to a minor inconsistency if, after the user has opened a page, the user submits a rating on another device, then submits a rating on the original device; the original device would not receive the rating from the other device. However, this inconsistency will be resolved the next time the original device needs to fetch rating data from the server.
Contents page
An advanced Contents page with many features is implemented in Special:Contents, and it is the target of the Contents link in the top navigation bar. A table of articles (with advanced sorting features) and a global page graph comprise the contents page.
The contents page has 3 parts: code common to both the graph part and the table part are in contents.js, code specific to the contents table is in contentsTable.js, while the global page graph code is in globalMap.js.
Dashboard and Profile pages
The Dashboard and Profile pages share the module ext.userDash.dashboard, as they share the donutProgessGraph.
- dashboard.js, used in Dashboard, creates a list of "suggestions" for articles for the user to visit next, and obtains the last article visited by the user for the "Continue" link.
donutProgressGraph.jsis shared by both the Dashboard and Profile, and it uses data from the page map to calculate and display a user's progress through learning the contents of the wiki.- ratingHistogram.js, used in Profile, uses data from the page map to tally the count of each vote a user has made, and displays the list of pages a vote is made on.
- ratingChart.js, used in Profile, fetches the entire rating history of a user from the server, then displays it in a chart of ratings/time. This functionally represents the full progress made by the user across time. Additionally, it can display the expected progress of a student throughout the year - however, as students are no longer expected to learn most articles in the wiki, this feature is disabled by default; to re-enable it, set the flag "UDEnableProgress" to true in extension.json.
Tests
A small amount of tests have been written, in UserDash/tests/qunit/ext.userDash.test.dataRepository.js, that tests the dataRepository's ability to convert data from MediaWiki into the page map object, updating the satisfiedness of an article, and the dashboard's ability to generate suggestions based on some sample data. The tests are by no means comprehensive, so improvements are welcome.
Notable decisions
- Pages are referenced by page names - specifically, the fulltext name, or mw.config.values.wgPageName.replace(/_/g, " ") - rather than ID in the extension. This means that page moves needs special handling (which has been implemented) to make the data work properly.
- For API calls, mw.config.values.wgPageName alone (as well as the fulltext form itself) works fine as the API code converts the title to fulltext form.
