Digital Measurement SEI Data Delivery Specification: Difference between revisions

From Engineering Client Portal

(New page added to cover data delivery specification)
 
No edit summary
Line 12: Line 12:
*** PARQUET format(.parquet) --- Preferred;
*** PARQUET format(.parquet) --- Preferred;
* All files should have extensions to indicate the file format (.json or .parquet)
* All files should have extensions to indicate the file format (.json or .parquet)
* Data file can be partitioned into multiple parts with min size of 128MB - 256MB
* Large data file can be partitioned into multiple parts with min size of 128MB - 256MB
* Data can be delivered separately in multiple splits(platforms), if needed due to organizational, technical or privacy requirements. This allows to permission s3 access separately, to process data independently and to persist data partitioned, within the limits of the S2S system;
* Data can be delivered separately in multiple splits(platforms), if needed due to organizational, technical or privacy requirements. This allows to permission s3 access separately, to process data independently and to persist data partitioned, within the limits of the S2S system;


Line 41: Line 41:


* The client's IAM Role ARN to grant access to Nielsen S3 bucket provided by client.
* The client's IAM Role ARN to grant access to Nielsen S3 bucket provided by client.
'''Onboarding steps to be performed by client'''


'''Onboarding steps to be performed by client'''
'''Onboarding steps to be performed by client'''
Line 47: Line 48:
# Attach the following IAM policy to the user or role to grant necessary permissions. This policy allows the user or role to read, write, and manage object access control lists (ACLs) in the specified S3 bucket. <code>{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:PutObjectAcl" ], "Resource": "arn:aws:s3:::<AccountABucketName>/*" } ] }</code> '''Note:''' Replace <code><AccountABucketName></code> with the bucket name provided by Nielsen. You can also restrict access to a specific folder by modifying the <code>Resource</code> element (e.g., <code>"arn:aws:s3:::AccountABucketName/FolderName/*"</code>). Check <nowiki>https://repost.aws/knowledge-center/cross-account-access-s3</nowiki> for more details.
# Attach the following IAM policy to the user or role to grant necessary permissions. This policy allows the user or role to read, write, and manage object access control lists (ACLs) in the specified S3 bucket. <code>{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:PutObjectAcl" ], "Resource": "arn:aws:s3:::<AccountABucketName>/*" } ] }</code> '''Note:''' Replace <code><AccountABucketName></code> with the bucket name provided by Nielsen. You can also restrict access to a specific folder by modifying the <code>Resource</code> element (e.g., <code>"arn:aws:s3:::AccountABucketName/FolderName/*"</code>). Check <nowiki>https://repost.aws/knowledge-center/cross-account-access-s3</nowiki> for more details.
# '''Share''' the '''IAM Role ARN''' to us.
# '''Share''' the '''IAM Role ARN''' to us.
Once you received S3 bucket information from Nielsen, perform the following steps to test access.


Once you received S3 bucket information from Nielsen, perform the following steps to test access.
Once you received S3 bucket information from Nielsen, perform the following steps to test access.

Revision as of 11:44, 6 August 2025


Introduction

This document outlines the data delivery specification for Ad and Content Measurement products via SEI integration.

Files should be delivered at a fixed cadence in a dedicated S3 bucket following the outlined in this spec prefix and object naming conventions and data formats.

  • All prefix labels and file names should be lowercase except for app ids.
  • Supported formats are:
    • Uncompressed text files with utf-8 encoding in:
      • JSON format(.json);
      • PARQUET format(.parquet) --- Preferred;
  • All files should have extensions to indicate the file format (.json or .parquet)
  • Large data file can be partitioned into multiple parts with min size of 128MB - 256MB
  • Data can be delivered separately in multiple splits(platforms), if needed due to organizational, technical or privacy requirements. This allows to permission s3 access separately, to process data independently and to persist data partitioned, within the limits of the S2S system;

Prerequisites

To get started, you will need a Nielsen App ID. The App ID is a unique ID assigned to your app. This will be provided to you by your assigned Technical Account Manager upon starting the integration.

Item Description Source
App ID (AppID) Unique ID assigned to the player/site and configured by product. Provided by Nielsen
Integration ID (IntID) Integration id provided by Nielsen. Used to link different integration inputs, ex. associate single audit ping integration with different viewership pipelines (dar, dtvr, dcr, etc.) Provided by Nielsen
Partner ID (PartnerID) Partner id provided by Nielsen. Used for the client specific S3 bucket .
Please note that the sample PartnerID 'acme' used in all examples is a placeholder to be replaced with your PartnerID. 
Provided by Nielsen

The onboarding process varies depending on the client's cloud environment.

AWS to AWS Integration

Prerequisites

  • The client's IAM Role ARN to grant access to Nielsen S3 bucket provided by client.

Onboarding steps to be performed by client

Onboarding steps to be performed by client

  1. Create an IAM user or role in client AWS account.
  2. Attach the following IAM policy to the user or role to grant necessary permissions. This policy allows the user or role to read, write, and manage object access control lists (ACLs) in the specified S3 bucket. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:PutObjectAcl" ], "Resource": "arn:aws:s3:::<AccountABucketName>/*" } ] } Note: Replace <AccountABucketName> with the bucket name provided by Nielsen. You can also restrict access to a specific folder by modifying the Resource element (e.g., "arn:aws:s3:::AccountABucketName/FolderName/*"). Check https://repost.aws/knowledge-center/cross-account-access-s3 for more details.
  3. Share the IAM Role ARN to us.

Once you received S3 bucket information from Nielsen, perform the following steps to test access.

Once you received S3 bucket information from Nielsen, perform the following steps to test access.

1. List Files (ls)

To see a list of the files in your designated folder: aws s3 ls s3://<bucket-name>/<prefix>/ To list all the files in all the sub folders as well: aws s3 ls s3://<bucket-name>/<prefix>/ --recursive Example: aws s3 ls s3://useast1-nlsn-w-dig-sei-partnera-feeds-prod/dar-exposure/ --recursive

2. Copy a Single File (cp)

To upload a single file from your local machine to S3: aws s3 cp yourfile.txt s3://<bucket-name>/<prefix>/

This command copies filename.csv to your s3 bucket at prefix folder.

3. Sync a Directory (sync)

To upload all new or updated files from a local directory to your S3 folder: aws s3 sync /path/to/your/local/directory s3://<bucket-name>/<prefix>/ This command uploads all the files in your directory to your s3 bucket folder.