DTVR SEI Viewing: Difference between revisions

From Engineering Client Portal

(Created page with "Placeholder")
 
No edit summary
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
Placeholder
{{Breadcrumb|}} {{Breadcrumb|Digital}} {{CurrentBreadcrumb}}This interface specification describes the file format that needs to be used to supply Nielsen with end-user viewing data captured by a “server-to-server” method for the Nielsen Digital TV Ratings (DTVR) product.
 
* The specification is defined using a JSON schema, with each file ideally containing all the viewing that has been captured within an hour.
* In the case of a stream that has one version with blackouts and one without, one with mobile and one non-mobile, each stream would have it’s own unique asset id.
* The audio file with related data should arrive before the viewing data file for the same hour.
[[File:S2S Diagram.png|thumb|700x700px]]
 
=== SLA ===
The files must be delivered into the proper S3 bucket within 3 hours of the start of that viewing file interval.  For example, files from 1:00 AM to 2:00 AM must be delivered before 4 AM.
 
=== File Naming Convention ===
File should be named with following convention: FileType_AppID _UTCStart_UTCEnd
 
File Type: DTVRViewingFile
 
AppID: app identifier
 
UTCStart (time in 32-bit unsigned int UTC in seconds from first second of audio captured)
 
UTCEnd (time in 32-bit unsigned int UTC in seconds from first second of audio captured)
 
Files should be placed in S3 buckets with the following folder structure:
 
[To be determined during design phase]
 
Nielsen will move files to another location as they are processed so processing status is apparent. Accuracy of measurement
 
These thresholds are currently under review during the Beta program period, and may change before data collected in this manner flows into Nielsen currency products.
 
The reported “wall clock” time of viewing needs to be within 25 seconds of the actual viewing
 
The reported content “reference” time needs to match the actual content that was played out with plus/minus 10 seconds (for clarity: start and end times can each be off by up to 10 seconds, so the combined under/over reporting for each individual viewing segment is no more than 20 seconds).
 
=== JSON Param Definition ===
'''Header Record'''
{| class="wikitable"
!'''Parameter'''
!'''Description'''
!'''Mandatory'''
!'''Specified by'''
!'''Format or Example'''
|-
|schemaversion
|Json Schema Version
|Yes
|Nielsen
|Example: S2SDTVRViewingV1.3
|-
|apid
|Server App ID
|Yes
|Nielsen
|Example: A487421B-XXXX-YYYY-8343-E3BBB66E44F2
|-
|type
|Type of data
|Yes
|Client
|Value: "content"
|-
|producttype
|Product Type
|Yes
|Client
|Value: "DTVR"
|-
|starttime
|File Start Time
|Yes
|Client
|Format: utc (time in 32-bit unsigned int UTC in seconds)
|-
|endtime
|File End Time
|Yes
|Client
|Format: utc (time in 32-bit unsigned int UTC in seconds)
|-
|recordcount
|# of records in body of file
|Yes
|Client
|Format: integer
 
Note: Multiple entries in position array still only count as a single record for the purposes of the file record count
|}
'''Header Record - Example:'''
<code>{"schemaversion":"S2SDTVRViewingV1.3","apid":"A487421B-XXXX-YYYY-8343-E3BBB66E44F2","type":"content","producttype":"DTVR","starttime":"1473781507","endtime":"1473791507","recordcount":"14"}</code>
 
 
'''Body'''
{| class="wikitable"
| colspan="2" |'''Parameter'''
!'''Description'''
!'''Mandatory'''
!'''Specified by'''
!'''Format or Example'''
|-
| colspan="2" |apn
|Client side App name
|Yes
|Client
|Format: alphanumeric, Example: BestAppIOS
|-
| colspan="2" |apv
|Client side App version
|No
|Client
|Format: alphanumeric, Example: 1.3
|-
| colspan="2" |sessionid
|Unique id generated when player is instantiated
|Yes
|Client
|Format: alphanumeric, Example: 16 char guid
|-
| colspan="2" |streamid
|id for every new instance of an asset
|Required if no sessionid
|Client
|Format: alphanumeric, Example: UTC or other random #
|-
| colspan="2" |sessionended
|Session is known to have ended in this file
|Yes
|Client
|Format: numeric,
 
Example: 1 (session will continue in next file), 2 (session closed)
|-
| colspan="2" |assetid
|id for an asset
 
(could be TMS id)
|Yes
|Client
|Format: alphanumeric, Example: VID123456789
|-
| colspan="2" |assetname
|Video asset name
 
used for troubleshooting
|Yes
|Client
|Format: alphanumeric, Example: “A&E High Def East” or GraceNote channel mnemonic or call station letters
|-
| colspan="2" |origcontent
|The origin of the content that is being played out
|No
|Client
|Format : integer
 
0 – (assumed default if no value provided); content is being streamed from source other than user's device e.g. Cloud DVR, MVPD VoD Library, or Publisher live-feed
 
1 – content is being played out directly from user device e.g. local DVR
|-
| colspan="2" |picturemode
|Picture in Picture Mode
|No
|Client
|Format: integer – 0 (no PnP), 1 (PnP being used)
|-
| colspan="2" |secondscr
|Second Screen Indicator. For Mirroring & Extended Screen via OTT device scenarios
|No
|Client
|Format: alphanumeric, Example: MIR or OTT
|-
| colspan="2" |position
|Array of contiguous content viewing
|Yes
|Client
|Format:
 
[{ "referencestart":"utc",
 
"referenceend":"utc",
 
"playheadstart":"utc",
 
"playheadend":"utc" "}]
 
Additional Notes:
 
<nowiki>*</nowiki> For viewing gaps less than 1 second, the gap can be smoothed over
 
<nowiki>*</nowiki> For playouts that are abandoned before the first whole second of viewing is rendered, a position record of zero duration should be created IF an audit ping has already been fired
|-
|{
|referencestart
|Wall clock reference start time
|Yes
|Client
|Format: utc (time in 32-bit unsigned int UTC in seconds)
|-
|
|referenceend
|Wall clock reference end time
|Yes
|Client
|Format: utc (time in 32-bit unsigned int UTC in seconds)
|-
|
|playheadstart
|Content position start time
|Yes
|Client
|Format: utc (time in 32-bit unsigned int UTC in seconds)
|-
|<nowiki>}</nowiki>
|playheadend
|Content position end time
|Yes
|Client
|Format: utc (time in 32-bit unsigned int UTC in seconds)
|-
| colspan="2" |device_id
|Mobile Ad ID (IDFA, ADID), Connected Device ID. Should not be the user ID.
 
If hashing required, SHA-256 is preferred
|Yes, for opted-in mobile devices where available
|Client
|A487421B-XXXX-YYYY-8343-E3BBB66E44F2
|-
| colspan="2" |hem_sha256
|SHA-256 hashed email
 
Note: email normalization rules applied before hashing
|Strongly Preferred
|Client
|55C06A30DAA5D5F382FDEB8C702EC57875CC9D91A7C78BB620053FD81DC4335C
|-
| colspan="2" |hem_md5
|MD5 hashed email
 
Note: email normalization rules applied before hashing
|Preferred
|Client
|3E168605F46DFB596EB717b7C83ACBC0
|-
| colspan="2" |hem_sha1
|SHA-1 hashed email
 
Note: email normalization rules applied before hashing
|Preferred
|Client
|37B7410CD605DB6294318CDB174014F062F0E9C6
|-
| colspan="2" |hem_other
|Hashed email if hashing algorithm is unknown or not sha256, md5, or sha1
|Preferred
|Client
|3E168605F46DFB596EB717b7C83ACBC0
|-
| colspan="2" |luid
|Living Unit ID - Experian Household ID
|Yes, for CTV where available, or audit ping
|Client
|B0EOFEDgD
|-
| colspan="2" |xff
|IP address
|Yes
|Client
|Format: xxx.xxx.xxx.xxx or xxx.xxx.xxx.000
 
or 000.000.000.000 for opt-out
|-
| colspan="2" |xdua
|Device User Agent
|Desired
|Client
|Format: alphanumeric, Example: Apple-iPhone1C2/801.293
|-
| colspan="2" |device_platform
|Device Platform
|Required if UA not shared
|Client
|Format: alphanumeric, Example DSK, MBL, OTT
|-
| colspan="2" |device_group
|Device Group
|Required if UA not shared
|Client
|Format: alphanumeric, Example PHN = phone, TAB = tablet, DSK = desktop, STV = Smart TV, UNWN = unknown
|-
| colspan="2" |os_group
|OS Group
|Required if UA not shared
|Client
|Format: alphanumeric, Example DROID, IOS, UNWN
|}
Note: All parameters are case sensitive.
 
'''Body - Example:'''
<code>{"apn":"BestAppiOS","apv":"1.1","sessionid":"123456789", "streamid":"123465435","sessionended":"1", "assetid":"vid345-67483","assetname":"ABC HD EAST","origcontent":"0","picturemode":"0","secondscr":"MIR","xff":"205.114.192.12","device_platform":"MBL","device_group":"PHN","os_group":"ios","xdua":"Apple-iPhone1C2/801.293","position":[{"referencestart":"utc","referenceend":"utc","playheadstart":"utc","playheadend":"utc"}</code>

Latest revision as of 15:39, 3 July 2024

Engineering Portal / Digital / DTVR SEI Viewing
This interface specification describes the file format that needs to be used to supply Nielsen with end-user viewing data captured by a “server-to-server” method for the Nielsen Digital TV Ratings (DTVR) product.

  • The specification is defined using a JSON schema, with each file ideally containing all the viewing that has been captured within an hour.
  • In the case of a stream that has one version with blackouts and one without, one with mobile and one non-mobile, each stream would have it’s own unique asset id.
  • The audio file with related data should arrive before the viewing data file for the same hour.
S2S Diagram.png

SLA

The files must be delivered into the proper S3 bucket within 3 hours of the start of that viewing file interval.  For example, files from 1:00 AM to 2:00 AM must be delivered before 4 AM.

File Naming Convention

File should be named with following convention: FileType_AppID _UTCStart_UTCEnd

File Type: DTVRViewingFile

AppID: app identifier

UTCStart (time in 32-bit unsigned int UTC in seconds from first second of audio captured)

UTCEnd (time in 32-bit unsigned int UTC in seconds from first second of audio captured)

Files should be placed in S3 buckets with the following folder structure:

[To be determined during design phase]

Nielsen will move files to another location as they are processed so processing status is apparent. Accuracy of measurement

These thresholds are currently under review during the Beta program period, and may change before data collected in this manner flows into Nielsen currency products.

The reported “wall clock” time of viewing needs to be within 25 seconds of the actual viewing

The reported content “reference” time needs to match the actual content that was played out with plus/minus 10 seconds (for clarity: start and end times can each be off by up to 10 seconds, so the combined under/over reporting for each individual viewing segment is no more than 20 seconds).

JSON Param Definition

Header Record

Parameter Description Mandatory Specified by Format or Example
schemaversion Json Schema Version Yes Nielsen Example: S2SDTVRViewingV1.3
apid Server App ID Yes Nielsen Example: A487421B-XXXX-YYYY-8343-E3BBB66E44F2
type Type of data Yes Client Value: "content"
producttype Product Type Yes Client Value: "DTVR"
starttime File Start Time Yes Client Format: utc (time in 32-bit unsigned int UTC in seconds)
endtime File End Time Yes Client Format: utc (time in 32-bit unsigned int UTC in seconds)
recordcount # of records in body of file Yes Client Format: integer

Note: Multiple entries in position array still only count as a single record for the purposes of the file record count

Header Record - Example: {"schemaversion":"S2SDTVRViewingV1.3","apid":"A487421B-XXXX-YYYY-8343-E3BBB66E44F2","type":"content","producttype":"DTVR","starttime":"1473781507","endtime":"1473791507","recordcount":"14"}


Body

Parameter Description Mandatory Specified by Format or Example
apn Client side App name Yes Client Format: alphanumeric, Example: BestAppIOS
apv Client side App version No Client Format: alphanumeric, Example: 1.3
sessionid Unique id generated when player is instantiated Yes Client Format: alphanumeric, Example: 16 char guid
streamid id for every new instance of an asset Required if no sessionid Client Format: alphanumeric, Example: UTC or other random #
sessionended Session is known to have ended in this file Yes Client Format: numeric,

Example: 1 (session will continue in next file), 2 (session closed)

assetid id for an asset

(could be TMS id)

Yes Client Format: alphanumeric, Example: VID123456789
assetname Video asset name

used for troubleshooting

Yes Client Format: alphanumeric, Example: “A&E High Def East” or GraceNote channel mnemonic or call station letters
origcontent The origin of the content that is being played out No Client Format : integer

0 – (assumed default if no value provided); content is being streamed from source other than user's device e.g. Cloud DVR, MVPD VoD Library, or Publisher live-feed

1 – content is being played out directly from user device e.g. local DVR

picturemode Picture in Picture Mode No Client Format: integer – 0 (no PnP), 1 (PnP being used)
secondscr Second Screen Indicator. For Mirroring & Extended Screen via OTT device scenarios No Client Format: alphanumeric, Example: MIR or OTT
position Array of contiguous content viewing Yes Client Format:

[{ "referencestart":"utc",

"referenceend":"utc",

"playheadstart":"utc",

"playheadend":"utc" "}]

Additional Notes:

* For viewing gaps less than 1 second, the gap can be smoothed over

* For playouts that are abandoned before the first whole second of viewing is rendered, a position record of zero duration should be created IF an audit ping has already been fired

{ referencestart Wall clock reference start time Yes Client Format: utc (time in 32-bit unsigned int UTC in seconds)
referenceend Wall clock reference end time Yes Client Format: utc (time in 32-bit unsigned int UTC in seconds)
playheadstart Content position start time Yes Client Format: utc (time in 32-bit unsigned int UTC in seconds)
} playheadend Content position end time Yes Client Format: utc (time in 32-bit unsigned int UTC in seconds)
device_id Mobile Ad ID (IDFA, ADID), Connected Device ID. Should not be the user ID.

If hashing required, SHA-256 is preferred

Yes, for opted-in mobile devices where available Client A487421B-XXXX-YYYY-8343-E3BBB66E44F2
hem_sha256 SHA-256 hashed email

Note: email normalization rules applied before hashing

Strongly Preferred Client 55C06A30DAA5D5F382FDEB8C702EC57875CC9D91A7C78BB620053FD81DC4335C
hem_md5 MD5 hashed email

Note: email normalization rules applied before hashing

Preferred Client 3E168605F46DFB596EB717b7C83ACBC0
hem_sha1 SHA-1 hashed email

Note: email normalization rules applied before hashing

Preferred Client 37B7410CD605DB6294318CDB174014F062F0E9C6
hem_other Hashed email if hashing algorithm is unknown or not sha256, md5, or sha1 Preferred Client 3E168605F46DFB596EB717b7C83ACBC0
luid Living Unit ID - Experian Household ID Yes, for CTV where available, or audit ping Client B0EOFEDgD
xff IP address Yes Client Format: xxx.xxx.xxx.xxx or xxx.xxx.xxx.000

or 000.000.000.000 for opt-out

xdua Device User Agent Desired Client Format: alphanumeric, Example: Apple-iPhone1C2/801.293
device_platform Device Platform Required if UA not shared Client Format: alphanumeric, Example DSK, MBL, OTT
device_group Device Group Required if UA not shared Client Format: alphanumeric, Example PHN = phone, TAB = tablet, DSK = desktop, STV = Smart TV, UNWN = unknown
os_group OS Group Required if UA not shared Client Format: alphanumeric, Example DROID, IOS, UNWN

Note: All parameters are case sensitive.

Body - Example: {"apn":"BestAppiOS","apv":"1.1","sessionid":"123456789", "streamid":"123465435","sessionended":"1", "assetid":"vid345-67483","assetname":"ABC HD EAST","origcontent":"0","picturemode":"0","secondscr":"MIR","xff":"205.114.192.12","device_platform":"MBL","device_group":"PHN","os_group":"ios","xdua":"Apple-iPhone1C2/801.293","position":[{"referencestart":"utc","referenceend":"utc","playheadstart":"utc","playheadend":"utc"}