XML Format

XML format

The XML feed is recommended as an alternative to the standard IPTC7901 wire message format and is superior in many ways. Customers are strongly advised to migrate to the XML service if they are currently using the wire formats for election data e.g. TTY or Dataformat

PA covers the various types of election events using different formats of XML that adhere to an XML schema. Unfortunately, at this time, there is not ‘one size fits all’ format of XML for all types of election events.

Delivery method

Delivery is by FTP Pull from PA’s servers.

To gain access, email customer.services@pressassociation.com, giving the IP address from which you are connecting. You will then receive the login details for live and test accounts to access the PA FTP servers.

You will need to use an FTP client to access the PA servers and download the files. Although this can be done manually, to become familiar with the message, in the long term this should be an automatic process in your system.

Please note that at this time PA does not offer an election API that permits data to be retrieved on demand. This will be reviewed in the future. However the XML service does provide a full range of data requirements and is always evolving due to customer feedback.

Customer technical requirements

The election XML is generated in real-time, so that it may be used to obtain the very latest results and state of election. Although the XML is human readable, customers are strongly advised to invest in the development of a system to download and ingest the XML automatically as it is generated. Only by doing this will the benefit of the real-time service be obtained. The following basic technical requirements are essential for the handling of the elections XML.

FTP Client capability: In order to login, list and download the XML files from the FTP servers, customers will need to use code libraries that provide standard FTP client capabilities. This common and usually available in most languages and development frameworks.
Tracking of files and their timestamps: Whilst is is feasible to download all files available on the FTP folders, most customers would want to keep track of the files used so far to increase efficiency and reduce downloads. It is therefore necessary to invest in some scripting to check for new files, keep track of timestamps and so on.
XML parsing capability: Since the data is in XML format, access to an XML parser is required to process the XML so that the values within are easily extracted. It is not necessary to validate the XML by the schema, which is costly, since it has already been validated during generation. Almost all development languages and frameworks offer easy to use APIs for parsing XML into usable document models.
XPath capability: To complement the parsing of the XML, XPath capability greatly assists in the extraction of values from the XML document. Most languages and frameworks have an XPath solution.
Database capability: It is necessary to aggregate much of the XML data in order to generate ‘overall’ views of an election, such as all results received so far. Unless a dedicated XML message is providing it directly, customers may need to aggregate the information provided by the XML into a database, so that they may extract this information for themselves. This could also be achieved by bulk processing all XML files in real-time.

Access to real election data

To access the XML generated from real election events you will need to use the live FTP account provided by customer services. Once logged into this account, the messages will appear in the following folders:

nominations Use this folder to download nominations data for all elections, in various XML formats. Nominations data will be published here well in advance of live election events. Use this data to set up your system for the processing of the live results data, such as constituencies, parties and candidates.Updates to the nominations will take place at regular intervals, to convey necessary changes and corrections. The PA will not change the real nominations on the day of an election unless it is absolutely necessary.The only way to obtain the correct IDs, names and abbreviations for a real election is to process the real nominations data – do not use the data from test elections via the test account!
results Use this folder to download live results data for all elections, in various XML formats. Results data will be published here during the course of live election events. Use this data to track each result and the overall state of an election.This folder will be empty before a real or test election event, and is gradually populated as results come in.

Access to test election data

To access the real-time test XML generated during scheduled election tests, please use the test FTP account provided by customer services. The test account will offer the same two folders as the live account, but they will be populated with test data generated from test elections.

Please check the advisories page for current information on which elections the PA is covering and the test rota.

Also, the XML produced from tests is always added to the XML Examples page. Here you may not only download zip files of previous tests but also archived data from previous real elections.

The election tests are designed to demonstrate the data carried by the XML messages, as we play out various possible election scenarios. These scenarios will vary from one test to the next. Please be aware that we never use real candidate names in our tests, although we do use the names of real parties and voting areas.

The IDs of entities provided in the election tests are NOT the same as those provided in the real data, such as for parties. Please do not configure your system for a real election event using the IDs provided in the test data. THEY WILL NOT BE THE SAME.

Please be aware that PA regularly housekeeps the files present on the test account, owing to the demands of the often tight election test schedule. Therefore the files from a test may only be available for a limited time. We always endeavour to provide the test data on the XML Examples page, if you have missed a live test.

In order to prepare for an election test, you will need to have taken the latest nominations data from the nominations folder of the test account. These nominations may change for every test, perhaps with completely different candidates. However we do try and keep the test data as stable as possible in terms of the voting areas and parties involved.

The test nominations will be uploaded to the test account at least on the day of each test, if not before.

The real-time results will be provided via the results folder of the test account. This will be empty at the start of a test and should be polled in real-time during a test in order to process the files that will appear.

Differences between live and test data

When conducting election tests, PA must use fictitious candidate names when simulating various election scenarios. Therefore data published in the nominations and results folders of the test account is strictly intended for test purposes only. It must not be confused with live data for real elections.

Unfortunately this means we cannot conduct tests using the real election data, which will only appear in the folders of the live account.

The IDs of any entities in test data, such as political parties, belong only to test elections. They will not be the same as entities found in the real election data, even if the names and descriptions are the same!

Please do not use the entity IDs in test data to set-up, prepare or configure the processing of real elections data available only from the live account.

To know the IDs of entities in the real election data, please use the nominations data published to the LIVE account. Only this will give you the IDs you need to set up your system to process the live data during the course of an election.

If you have any questions about the above or need further assistance, please contact PA customer services.

Polling the FTP folders for new files

It is necessary to poll the data folders on a regular basis for new files. Although all files could be downloaded each time, it is more efficient to keep track of the current list of files and most recent file timestamp to see what is new. You may then download only what is needed.

PA recommends that customers do not poll the folders more regularly than every 10 seconds. Unfortunately FTP does not tend to display timestamps above one minute accuracy, even though the files may have been updated within the last minute.

In order to not exceed account quotas and reduce load, it is not recommended that customers establish multiple concurrent connections to download the available files. It should still be expedient to list and download the files over a single connection.

To facilitate a more efficient approach, a ‘timestamp’ file is updated in each of the folders whenever a new XML file is deposited. The timestamp file is called:

.timestamp

This file contains the number of seconds since the epoch, with millisecond accuracy, such as “1405942919.30267”. By regularly polling this file and using its modification time, or the epoch value within, a consumer can decide when to fully scan the folder for new XML files. When the timestamp file has been updated, new files should be looked for. It would be prudent to still scan the folder for new files periodically, but not as frequently as the polling of the timestamp file.

Filenames

Elections XML messages generally follow a filename pattern that provides the election name, the message type, the constituency or area, and possibly a revision number e.g.

General_Election_result_Witney_1.xml

Messages generated in tests, where a test election has been used as a source, will have the word Test prefixed or suffixed to the election name.

Making use of PA IDs for candidates and parties

The XML message feed contains an enhancement which is not available in the other feed formats, namely unique ID numbers for each candidate and party. These can be relied upon to match candidates from nominations messages to results messages, and to uniquely identify different independent candidates. They cannot be used to match candidates between elections, though: due to the complexity of identifying that candidate J. Smith standing in constituency X in 2015 is the same J. Smith standing in constituency Y in 2017, PA’s software does not attempt to make that link, so the candidate IDs will actually be different in the two elections. For similar reasons, party IDs are also different between different elections.

The IDs used in tests will be different to the IDs used in real elections, although names and abbreviations may be the same.

Note that in the case of mayoral elections each poll is run as a separate entity. Where there are multiple mayoral polls, the same party will have a different party ID in each one. Customers must ensure they use the IDs specific to each mayoral election, obtainable from the nominations files.

Message types available

PA groups elections XML messages by the voting system in use. This permits a variety of election events to be covered by using the most appropriate message type. The XML messages are divided into the followed types by voting system:

First Past the Post (FPTP)
Proportional Representation Top-up (PR Top-up)
Local Council
Single Transferable Vote and variants thereof (STV)
Referendums

First Past the Post and PR Top-up

The FPTP voting system is characterised by the winning candidate being the one who has received the most votes in a ballot. Furthermore, the winning party overall is determined by the total number of seats won, which may or may not be a majority. A majority is needed for the immediate formation of a governing body. The Press Association’s handling of FPTP elections is principally focused on the mechanics of General and Assembly elections in the UK.

The FPTP messages are used to cover General, Scottish Parliament, London Assembly and Welsh Assembly elections, the following types of data are available in the XML:

Nominations
Rushes
Results
Recount notifications
State of parties, including forecasts

The PR Top-up voting system is used for top-up constituencies in assembly and European elections. It is typically used alongside the usual FPTP voting in regular constituencies. The following types of message are available in XML:

Nominations
Rushes
Results

Getting started with FPTP and PR elections

To prepare your systems for FPTP or PR elections, you will probably want to download the nominations files first from our server. There is a nominations file for each constituency or voting area, giving the details of the constituency and all the candidates standing for election there.

Critically, these nominations messages provide the IDs of all the parties standing candidates.

There may also be some additional PR Top-up seats for a mainly FPTP elections, which will be handled using the appropriate XML messages

Each nominations message has a revision number, which is present in both the filename and the message itself. Whenever nominations for a constituency are updated, a new message with a higher revision number will be made available on the server. Note that a higher revision number does not necessarily mean that the nominations have actually changed, however.

It is likely that the previous lower revisions of a nominations message will remain on the server, so it is important that only the highest revision of each constituency is processed.

Occasionally corrections to nominations come to light only during the results operation, where suddenly we learn that there is an extra candidate, or that one has dropped out, or that a name was incorrect. In such a case, a revised nominations file for the affected constituency will automatically be made available, before the results.

These nominations messages now contain an optional ballotName element within the Candidate element for each nominated candidate. This optional, extra data, is designed to provide a more formal representation of the candidate name – as seen on a ballot paper.

The ballot name is comprised of the surname, then a comma, followed by no more than two space separated first names. The first name will be the more formal version as opposed the ‘known as’ first name normally used e.g. Peter rather than Pete. If the candidate has no first names only the surname will be provided in this attribute. If the candidate is historical and is missing this data, the ballotName attribute will be absent.

Important note for customers making use of the previousSittingMember element in the nominations files for set-up:

Generally speaking, the sitting party matches the party of the sitting MP (where there is a sitting MP) at the dissolution of Parliament – but there are a few exceptions. These are the cases of differences in PA’s live election settings for the 2019 General Election:

Beaconsfield: sitting party is Independent but sitting MP Dominic Grieve appears with party description Independent Dominic Grieve (abbreviation Ind)
Birkenhead: sitting party is Independent but sitting MP Frank Field is standing for Birkenhead Social Justice Party
Birmingham Hall Green: sitting party is Labour but sitting MP Roger Godsiff is standing as Independent
Bury South: sitting party is Independent but sitting MP Ivan Lewis appears with party description Independent Ivan Lewis (abbreviation Ind)
Luton South: sitting party is Independent but sitting MP Gavin Shuker is standing as a No description candidate

The cases of Beaconsfield and Bury South are accounted for by PA’s application of differentiation where there are multiple independents standing in the same constituency.

Note that at Eastbourne, PA has the sitting party as Liberal Democrat even though sitting MP Stephen Lloyd was sitting as Independent at the time of dissolution and only later returned to the Lib Dems.

Results for FPTP and PR elections

Each first-past-the-post result message gives the full details of the results in a constituency, with the numbers of votes, turnout, party shares and changes in party shares since the last election. Corrections may be sent to results messages, and these are indicated by increased revision numbers. The first version of any result has the revision number 1.

A first-past-the-post result is preceded by a rush message, which lists the candidates (corresponding to the nominations message for that constituency) and states which one has been elected. The rush message also contains the piece of text which goes out on the wire. Each rush message also has a revision number, which is the same as the number of the result message associated with it. When a correction is made to a result, a new rush message is only generated if the winning party in the correction differs from that originally announced.

Always use the rush and results messages of the highest revision for each constituency.

Both nominations and results messages also contain a summary of the previous election results in the constituency.

These messages now contain an optional ballotName element within the Candidate element candidate. This optional, extra data, is designed to provide a more formal representation of the candidate name – as seen on a ballot paper.

More detailed documentation on FPTP and PR Top-up messages can be found here.

Local Council

Although these polls use the FPTP system, these elections are only covered at the seats per party level within each council. We do not record the names of individual candidates or wards. However nominations are still available to assist in the setting up of councils, such as how many seats are being offered for election.

For local council elections, the following message types are available:

Nominations (as numbers of seats per party across the council – we do not record the names of individual candidates or wards)
Results (in the same style as the nominations)
State of parties

Getting started with local council elections

A set of XML nominations files will be made available on the server ahead of the election results operation. PA does not collect the names of individual candidates in local council elections, so by nominations we only mean the numbers of seats held, seats offered, unopposed returns and candidates standing for each party. This information can be used to prepare systems in anticipation of the results coming out.

Sometimes corrections to nominations may come to light only during the results operation, due to incorrect information having been provided to PA by local authorities. When this occurs, a revised nominations file for the affected council should appear on the FTP server. It will have an up-to-date timestamp, but currently nominations files do not contain a revision number.

Results for local council elections

Results messages will be published to the results folder when they are received.

In case of any corrections, results messages contain a revision number, both in the message content (as an attribute to the root element) and in the filenames.

State-of-parties messages are transmitted after every result. The number of a state-of-parties message is included in its filename as well as in the message itself.

More detailed documentation on local council messages can be found here.

Single Transferable Vote/Supplementary Vote

The Single Transferable Vote and the Supplementary Vote systems are characterised by election counts that can span more than one round of voting. Excess votes from previously elected or eliminated candidates may be transferred to other candidates in later rounds.

PA uses an STV type of message to cover Northern Ireland Assembly elections and Mayoral and Police Commissioner elections.

These are the types of message generated these types of election:

Nominations
Rushes
Results
State of parties, including forecasts
Recount notification

For police commissioner polls and mayoral elections in London and other cities, the following message types are available:

Nominations
Rushes
Results

Getting started with STV elections

Nominations for the STV variety of elections are published to the nominations folder. Should any corrections be necessary, these will be published before any result is produced.

Results for STV elections

The results of the Northern Ireland Assembly and various police and mayoral elections come out on a per-round basis. The results message for each numbered round includes a revision number both in its body and in its filename. Any corrections to earlier messages will be indicated by a new message appearing with a higher revision number.

Before the full votes count is known for each candidate in a round of results, a rush message may be generated detailing which candidates have been elected in that round, but containing no more than that. The full result that follows provides the votes and status of each candidate in that round.

Any previous rounds’ results are included in each results message, so every message should be fully usable in itself without having to archive the data from previous ones.

Referendums

Like the other systems, these polls are divided into discrete voting areas. However votes are made for each answer to a proposition, with the winning answer having the most votes. The overall winning answer is again the one receiving most votes.

For referendum polls, the following message types are available:

Rushes
Results
Recounts
Running totals

Getting started with referendum polls

A referendum may consist of one or more propositions, each of which has two or more answers that may receive votes. For each area covered in a referendum event, the messages supply the chosen answers and their votes for each proposition.

The referendum rush message simply conveys the winning answer(s) for each proposition, for a given area. At this point the votes and majority are not known, so the XML simply supplies the winning answer text in the winningAnswerText attribute of the Proposition element. In the case of a tie between two or more answers, this value will be “Tied”. The enclosed Answer elements also contain a ‘winning’ attribute that is set to true if that answer has been declared as having the most votes. In the case of a tie there will be two or more answers having this value set to true.

Results for referendum polls

The referendum results messages follow the same rules as above except that the votes and majority values are also supplied. In the case of a tie the votes counts will clearly be seen to be equal for two or more answers.

The results messages always supersede the rush messages, and should be used in preference once received. Should there be a change to the winning answers or votes for an answer, the rush and results messages will be repeated but with an incremented revision number (the last part of the XML filename and also a revision attribute in the root element).

Any test XML produced by PA referendum tests, will have the suffix “Test” added to the shortText attribute of the Answer elements in both the rushes and results. You may need to configured your system to expect this suffix when parsing the XML, to help identify each answer. Messages produced for a live event will not have this suffix, and you should configure your system to expect this accordingly.

The referendum running totals message conveys the votes share and percentage gained by each answer in a proposition. It also provides a running total of areas that have voted for that answer to win. In the case of a tie break over two or more answers, the ‘win’ count of areas will not be incremented against those answers, although the votes and percentage figures will still be increased (since they were still gained). Instead a “Tied” element will appear within the proposition, providing the number of areas that have tied. The sum of areas that are voted for each answer to win, and those tied, will be equal to the number of results received so far.

More detailed documentation on the referendum messages can be found here.

W3C schemas

All message types mentioned above are specified by a set of W3C schema files.

These have been updated, as of 9th May 2017, to include an optional ballotName attribute within the Candidate element of First Past The Post and PR Top nominations, rush and results XML. Please be aware that historical XML examples will not contain this attribute, although those examples will be compatible with the updated XML schemas.

Click here to download a ZIP file, which contains the following files:

DataTypes.xsd – essential prerequisite for the other schema files.
FirstPastThePostNominations.xsd – the schema for nominations in General Elections and to first-past-the-post constituencies in the Scottish Parliament, Welsh Assembly and London Assembly. Now including an optional ballot name.
TopUpNominations.xsd – the schema for nominations to top-up constituencies in the Scottish Parliament, Welsh Assembly and London Assembly. Now including an optional ballot name.
FirstPastThePostRush.xsd – the schema for rush messages in General Elections and in first-past-the-post constituencies in the Scottish Parliament, Welsh Assembly and London Assembly. Now including an optional ballot name.
TopUpRush.xsd – the schema for rush messages in top-up constituencies in the Scottish Parliament, Welsh Assembly and London Assembly. Now including an optional ballot name.
FirstPastThePostResult.xsd – the schema for results in General Elections and in first-past-the-post constituencies in the Scottish Parliament, Welsh Assembly and London Assembly. Now including an optional ballot name.
TopUpResult.xsd – the schema for results in top-up constituencies in the Scottish Parliament, Welsh Assembly and London Assembly
FirstPastThePostStateOfParties.xsd – the schema for state of parties messages in General Elections and in first-past-the-post constituencies in the Scottish Parliament, Welsh Assembly and London Assembly. Now including an optional ballot name.
Recount.xsd – the schema for messages announcing a recount in a constituency of a General Election or a a Scottish Parliament, Welsh Assembly or London Assembly election.
LocalNominations.xsd – the schema for local council nominations.
LocalResult.xsd – the schema for local council results.
LocalStateOfParties.xsd – the schema for local council state-of-parties messages.
ReferendumRush.xsd – the schema for referendum rushes.
ReferendumResult.xsd – the schema for referendum results.
ReferendumRecount.xsd – the schema for messages announcing a recount in a voting area in a referendum. Note that the root element is now ReferendumRecount, no longer the same as for assembly recount messages, which use Recount.
ReferendumRunningTotals.xsd – the schema for running totals in a referendum.
STVNominations.xsd – the schema for nominations in an election which uses Single Transferable Vote or a variation thereof, which includes mayoral elections in London and other English cities, Northern Ireland Assembly elections, and elections for the heads of police forces.
STVResults.xsd – the schema for rushes and results in an election which uses Single Transferable Vote or a variation thereof (see above for the list of elections to which this relates).

Queries and comments

If you have any technical queries or comments relating to the XML message feed, please contact PA Customer Services, and they will quickly pass on the message to the relevant person.