Newsagent/Tech

Newsagent
Author	Chris Page
Contact	Support
Repository	github
Issue tracker	github
OS	Neutral (web based) Will not work with IE8 or older
URL	Newsagent
Navigation
General Overview Feed List Feed Subscription Logging in Logout Articles Composing Articles (Quickstart) Image Library Files Notification Method Options Feeds and Visibility Multiple Feeds Listing Articles Editing Articles Template Articles Syndication Newsletters Newsletters Newsletter Editor Tell Us Tell Us Queues Tech docs Advanced/Upcoming features Installation Technical Autogenerated docs

Importer

This section contains some notes regarding the importer system and the steps importer modules should follow.

General structure

The importer consists of a view/controller entrypoint module import (Newsagent::Import in blocks/Newsagent/Import.pm), a base model module Newsagent::Importer in modules/Newsagent/Importer.pm. Newsagent::Importer dynamically loads subclasses (of the form Newsagent::Importer::<SourceName>, and here simply referred to as "sources" as a shorthand) to handle the importing of articles on a source-specific basic. Dynamic loading is done using through the system-wide Webperl::Modules::load_module(), and the source modules must implement the import_articles() method to do any work they need to do.

Invocation and execution flow

The importer is executed by loading the import module - no auth is needed for this module, and it can be used via http or https: https://newsagent.cs.manchester.ac.uk/import/<source> Here source can be one of:

• all - indicates that the importer should load and run all sources that need to run.
• a source name, corresponding to a row with a given shortname in the news_import_sources table to run a specific source's import.

The import module takes a number of steps:

• it checks the source is valid
• if the source if valid, it checks whether it is time to run the import for that source yet
• if it is time, the source module is loaded and its import_articles() is called.

(when the <source> argument is all, the above three steps happen for each defined source)