Data Provisioning Service (DPS): Supplementary Guidance and How to use the DPS test service
v2.6
Page 1 of 12
15/08/2017
1. How to use the DPS service The DPS API Specification published within the technical specifications contains instructions on the operation of the Data Provisioning Service. Development will involve: programmatically posting an XML call for the DPSrequestToken message using the WSDL file dpsauthentication.wsdl. This will return a security token to be passed with all subsequent method calls within a 4 hour session then programmatically posting XML method calls (including the token retrieved from the authentication call) for the following: DPSquery - this is the method called to obtain the number of data items waiting to be retrieved. DPSretrieve - this is the method called to retrieve data from the DPS test service. DPSdate2index - this is the method used to convert a calendar date, meaningful to the employer or agent, into a high water mark prior to that date. You will receive an appropriate XML response for any particular method call as defined in the DPS API Specification or the Response Error Messages document.
2. WSDLs and submission URLs For DPSrequestToken calls, the technical specifications contain a WSDL file named dpsauthentication.wsdl. The <soap:address> within this WSDL file should be changed as follows in order to make calls to the Third Party Validation Service (TPVS) and live environments: TPVS test service: <soap:address location="https://www.tpvs.hmrc.gov.uk/dpsauthentication/dpsauthentication.jws” /> Live: <soap:address location="https://dps.ws.hmrc.gov.uk/dpsauthentication/service" /> For subsequent method calls, the technical pack contains a WSDL file named dps.wsdl. The <soap:address> within this WSDL file should be changed as follows in order to make calls to the TPVS test service and live environments: TPVS test service: <soap:address location="https://www.tpvs.hmrc.gov.uk/dps/dps.jws" /> Live: <soap:address location="https://dps.ws.hmrc.gov.uk/dps/service” />
v2.6
Page 2 of 12
15/08/2017
3. Schemas The technical pack contains a schema for each of the message types that can be retrieved from the DPS. These define the structure and content of the outbound XML messages. In a small number of instances you may find the data contained within fields does not match the specific patterns included within the published schemas. The field names and structure of the outgoing messages retrieved from the DPS will be compliant with the published schemas. DPSretrieve calls: PAYE For DPSretrieve calls with a dataType of P6, P9, SL1 or SL2, the responses will conform to only one corresponding schema in the ZIP archive “Outgoing schemas”. AR, NOT and RTI DPSretrieve calls each have more than one schema associated with the responses. For dataType RTI, where there is data to retrieve, the responses will conform to:
RTINotif, in the ZIP archive “Outgoing schemas”, or the schemas in the “Generic Notification schemas”, or a mixture of the two.
DPS-enabled software will, therefore, need to be able to recognise and handle responses containing a mixture of RTI Notifications and Generic Notifications. DPSretrieve calls: Construction Industry Scheme (CIS) DPSretrieve calls with a dataType of CIS will conform to the schemas in the “Generic Notifications schemas”.
4. Stylesheets P6, P6B, P9, SL1, SL2, AR, NOT or RTI messages retrieved from the DPS can be viewed and/or downloaded in XML format. The technical pack contains XSLT stylesheets that allow the P6, P6B, P9, SL1, SL2, AR and NOT forms to be rendered into a human readable format. Stylesheets are not available for the RTI messages or for any Generic Notifications.
5. Response Error Messages If a DPS call fails validation, the error will be returned as a Soap fault. These faults are described within the Response Error Messages document within the technical specifications.
v2.6
Page 3 of 12
15/08/2017
6. RTI Notifications RTI Notifications using RTINotif-vN-N.xsd The RTI Notification messages will include a FormType attribute of “NVR” to identify whether the Notification originated following submission of a NINO Verification Request (NVR) or “NOT” if the Notification was generated following submission of an Employer Alignment Submission (EAS) or Full Payment Submission (FPS). This will allow third-party software to filter the responses, if necessary, to allow them to be matched to the incoming submission. Element/Attribute RTINot
Cardinality 1..1
@IssueDate
1..1
@FormType
1..1
@SequenceNumber CorrelationID
1..1 0..1
EmployerRef
1..1
Name
0..1
Name/Ttl Name/Fore Name/Initials Name/Sur NINOProvided
0..1 0..1 0..1 1..1 0..1
PayId
0..1
NINOToUse MessageID
v2.6
0..1 0..1
Description This is the root XML element containing the generic notification. Each call to DPS can return multiple RTINot messages within a single header. Load date of the notification. “NVR” for Notifications generated from a NINO Verification Request message. “NOT” for Notifications generated from a Full Payment Submission (FPS) or Employer Alignment Submission (EAS). Sequence number The CorrelationID allocated by the Transaction Engine to the incoming NVR, FPS or EAS message which instigated the Notification. This will allow the receiving software to associate the response with the request. The employer reference – composite of District number and reference. The parent element containing the name of the individual to whom the notification relates. May optionally contain a title May optionally contain forename details May optionally contain initials Surname Will be populated with the NINO which was provided on the ‘incoming’ message, if any For NVR messages this will contain the payroll ID included within the incoming NINO Verification Request. For NOT messages this will contain the payroll ID held by HMRC. Will be populated for messages 1, 3 Denotes the action to be taken in order to deal with the relevant Notification correctly. The table below states the current allowable values.
Page 4 of 12
15/08/2017
MessageID values and associated text Message FPS/EAS
NVR
FPS/EAS NVR
NVR
FPS/EAS NVR NVR NVR
Scenario Substitute the NINO provided on FPS/EAS for this NINO Substitute the NINO on the NVR for this NINO Stop using the NINO provided on FPS/EAS NINO provided on the NVR is incorrect and no NINO found No NINO provided on NVR and no NINO found Start using this NINO NINO found The NINO provided is correct Service currently unavailable
MessageID 1
Static Text This notification shows the correct NINO. Please use this NINO for any future submissions instead of the one you originally provided.
2
There was no NINO provided or the one given is incorrect. Please do not use the incorrect one for any future submissions
3
For future submissions please use the NINO provided in this notification The NINO you have provided is correct. For future submissions please continue to use this NINO. Service currently unavailable
4 5
In the Production environment, ‘filing only’ agents are able to make submissions through the Transaction Engine for clients for whom they have not been authorised to act. Agents cannot download Notifications from the DPS if they are not authorised to act on behalf of the employer. This could mean that an Agent will not be able to download NINO Notifications from the DPS even if these were generated following submission of a NINO Verification Request message.
v2.6
Page 5 of 12
15/08/2017
7. Generic Notifications Generic Notification Service (GNS) using GenericNotification-vN-N.xsd The generic notification message structure has been designed to support the following scenarios:
Notifications presenting information. o
E.g. Reminders, instructions, warnings, events
Notifications identifying data changes.
E.g. Instructions to use different data item values, identifying updates to data item values. The schema supports these scenarios and the requirement to support simple presentation to users as well as providing identifiers that 3rd party applications can choose to use in order to implement more complex/automated processes with or without user intervention. The schema supports these scenarios with the following properties: o
Relates to the employer reference and notifications may be at employer/contractor level or at a lower level, such as a specific appeal, or an individual employee.
May be in response to a correlating submission, or be unsolicited, or correlate to some other event.
Can optionally present any data item for information and/or change that can be mapped into a standard Key-Value pair structure and provides an extension point that can support more complex structures, if required in the future.
v2.6
Page 6 of 12
15/08/2017
7.1 – Generic Notification Structure Element/Attribute GenericNotification
Cardinality 1..1
@IssueDate
1..1
@FormType @SequenceNumber
1..1 1..1
MessageReference
1..1
MessageTypeID
1..1
MessageTypeTitle
1..1
MessageText
1..1
Description This is the root XML element containing the generic notification. Load date of the notification. This will be a hard coded enumeration of ‘GEN’. Notification sequence number used to control notification retrieval A globally unique randomly generated reference number for the notification message. For internal HMRC use only, not for display purposes. Each generic notification message type will contain a numeric identifier. Example messages are: 1 – Non filing notice 2 – Late filing notice 14 – CIS Penalty appeal accepted Each of the messages may include further data within the of the message. This will be included within an section identified by an appropriate namespace. Human readable title of the notification. See “Content of key-value pair structure” below for example content. The message text provided by HMRC which should be displayed to the user. This may contain newline characters to denote a new paragraph. The newline character is intended to be represented by the ASCII 'LF' character (Hex value 0A): http://en.wikipedia.org/wiki/ASCII Binary 000 1010
DutyType
1..1
EmployerRef
1..1
EmployerName Content
1..1 0..1
v2.6
Oct
Dec
012
10
Hex
Abbr
[a]
[b]
[c]
0A
LF
␊
^J
\n
Name Line feed
The text of each message should not exceed 1500 characters. This will be ‘PAYE’ for all RTI Generic Notifications, or ‘CIS’ for CIS Generic Notifications The employer reference – composite of District number and reference. The name of the employer or contractor. When included, will include content identified by namespace. May contain multiple items, but all items will be of the same type and namespace.
Page 7 of 12
15/08/2017
7.2 – Information Structure Element/Attribute Information
Cardinality 1..1
@seq
1..1
InformationItem
1.. unbounded 1..1
@id
DisplayName
Value @Type
1..1
1..1 1..1
Description This is the root XML element for the Information schema that extends from the Generic Notification ‘Content’ element. Integer sequence number to uniquely identify an iteration of this item. May contain one or more items which describe information related to this notification. ID of the data item number appearing within ‘Value’ See “Content of key-value pair structure” below for example content. The display name is a human readable description of the ID attribute. See “Content of key-value pair structure” below for example content. The information value e.g. 2013-04-05 An attribute of ‘Value’ to define the type of value and format to assist software developers e.g. ‘Date’, ‘String’ etc. See “Content of key-value pair structure” below for example content.
Content of key-value pair structure Each generic message will contain human readable text within the <MessageText> element. In addition to this text, message-specific data can be contained within the section included within the of the message. Examples are given in the table below. MessageTypeID 1
MessageTypeTitle Non filing notice
@id 301
DisplayName Period Ending
Value YYYY-MM-DD
@Type xsd:date
2 14
Late filing notice CIS Penalty appeal accepted
300 305
Date of Receipt Unique ID
YYYY-MM-DD String
xsd:date xsd:String
306
Default period ending
YYYY-MM-DD
xsd:date
307
Employer Accounts Office Reference
String
xsd:String
Please note that this list is not exhaustive: additional notifications may be added in future releases.
v2.6
Page 8 of 12
15/08/2017
7.3 – Change Structure The Change Structure is reserved for future use. Element/Attribute Change
Cardinality 1..1
@seq
1..1
ActionItem @id DisplayName PreviousValue
1.. unbounded 1..1 1..1 0..1
@Type
1..1
Value @Type
1..1 1..1
InformationItem
0.. unbounded 1..1 1..1 1..1 1..1
@id DisplayName Value @Type
Description This is the root XML element for the Change schema that extends from the Generic Notification ‘Content’ element. Integer sequence number to uniquely identify an iteration of this item. May contain one or more items which describe changes related to this notification. ID of the data item number appearing within ‘Value’ A description of the data content within ‘Value’ May optionally contain the previous information value e.g. 2012-09-07 An attribute of ‘PreviousValue’ to define the type of value and format to assist software developers e.g. ‘Date’, ‘String’ etc. The information value to be used e.g. 2012-09-14 An attribute of ‘Value’ to define the type of value and format to assist software developers e.g. ‘Date’, ‘String’ etc. May optionally contain one or more items which describe information related to this notification. ID of the data item number appearing within ‘Value’ A description of the data content within ‘Value’ The information value e.g. 2012-09-14 An attribute of ‘Value’ to define the type of value and format to assist software developers e.g. ‘Date’, ‘String’ etc.
8. Outgoing XML Generator (OXG) This tool allows you to create additional test data in XML format; this can be used to supplement the static data available for retrieval from the DPS test service. The OXG contains a number of .csv files which are already populated with dummy data. We recommend that you replace this with more meaningful data. Do not rely on the existing data in these templates for testing purposes. Any amendments to the individual .csv files contained within OXG should be made in notepad or txt format and then re-saved as a .csv file. If amendments are made in .csv format you may receive format errors when using OXG to convert the file into XML format. Generic Notifications may contain newline characters in the MessageText to denote a new paragraph (see section 7.2 above). When using the OXG to create Generic Notifications for test purposes, such newline characters can be replicated by inserting {newline} where required in the appropriate ‘MessageText’ field in the example template.
v2.6
Page 9 of 12
15/08/2017
9. Test service All calls to the TPVS test service should be made using the URL endpoints described in section 2 within this document. The DPS test server holds a pre-defined set of test data in respect of P6, P6B, P9, SL1, SL2, Annual reminders, P35, P11D(b), RTI Notifications and Generic Notifications which will be stored in the test service. The test service requires you to include your existing PAYE or CIS test credentials (SenderID and Value) to be included within the DPSrequestToken call (as Username and Password respectively). If you do not have credentials for use with the test service, please contact the Software Developer Support Team. In order to retrieve test data, the test service requires one of the following references in the <entity> element of your Soap call: 123/1739465, or 123/A6, or 123/AN64312 (not CIS), or 123/ZZYYXXWW (CIS only). To access all notifications in the test service, you will need to make retrievals for all these references. If you use references 848/I445 or 951/B7440 you will receive a response containing no forms. The test service has been set up as for an ‘agent’. Thus you can test retrieval of messages for multiple references if required. To do this, simply leave the <entity> element empty in your DPSretrieve call. If a request is made with a reference other than those listed, the test service will respond with an error indicating that the user does not have the rights to see data for that employer reference. You will have unrestricted access to the test service and testing can be managed according to your own cycles. All supporting documents required can be accessed at the DPS technical specifications page. RTI Notifications The NINO Verification Request message allows a user to request up to 100 NINOs to be verified within a single NVR message submitted through the Transaction Engine. The result of this NVR message will be made available for retrieval using DPS. In the live environment it is possible that not all of the NINOs included within a single submission will be matched within the same timeframe and therefore not all of the responses will be available for retrieval from DPS at the same time. It is not possible to replicate this behaviour within the test environment, as the test service contains static fictitious data which is not generated from data contained within an incoming submission. In the live environment there will be a genuine included within each NINO Notification to allow software to match it with the submitted NVR or FPS message which instigated the Notification. The test service data will contain fictitious entries within the element as these will not be generated from data contained within incoming submissions.
v2.6
Page 10 of 12
15/08/2017
Generic Notifications The test data for the GNS will include examples of generic wording within the <MessageText> element. This may not be an identical representation of the wording that will be issued to your customers but will include the examples of formatting in accordance with the live service.
10.
Supplementary Guidance
Content of SAML tokens The content of SAML tokens returned by the live service may look somewhat different to the content of the SAML tokens returned by the test service. However, since the SAML token is always regarded as an opaque ‘fragment’ of XML that is retrieved initially and then used unchanged in subsequent DPS calls, these differences should not affect the application code or the correct operation of the service. Please also note that there are some superficial differences between test and live service WSDLs relating to the exact strings used as namespace prefixes. The XML Namespace specification permits different name prefixes to be used in any particular XML document as long as the namespace ‘names’ to which the prefixes refer are identical. All element namespaces have been preserved between test and live services in both the WSDLs and any returned messages.
v2.6
Page 11 of 12
15/08/2017
Changes Version number 2.0
2.1
2.2 2.3
2.4 2.5
2.6
v2.6
Changes
Date
Document renamed (Previously: How to use the DPS test service and gain internet recognition V1.7) Rewritten to include RTI data Vendor Recognition section removed to separate document Supplementary guidance incorporated Changes section added Section 2: removed references to endpoint URLs https://ws.hmrc.gov.uk/dpsauthentication/service and https://ws.hmrc.gov.uk/dps/service which are no longer supported Section 7: replaced “Message No” with “MessageID” to match schema Section 7: Scenario 5 wording amended Section 7: RTI Notifications sub-divided into 7.1 for NINO Notifications and 7.2 for Generic Notification Service. Explanatory table added at 7.1. Section 9: para 7 amended to clarify that the CorrelationID will also be returned for Notifications generated from EAS or FPS, as well as NVR. Sections 3, 4, 7, 8, 9: updated re Generic Notifications. Section 9: new paragraph on retrievals for multiple references Minor changes to grammar and formatting for clarity Amended for the introduction of Generic Notifications for the Construction Industry Scheme. Section 6 removed, subsequent sections renumbered. Some grammar corrections. References to Government Gateway changed to Transaction Engine. Minor amendments for clarity.
24/06/2011
Page 12 of 12
24/10/2011
05/01/2012 31/10/2012
26/09/2014 26/03/2015
15/08/2017
15/08/2017