Use the compressedBinaryDataFileId property

This is for:

Developer System Administrator

Use the compressedBinaryDataFileId property when you want to push 256 MB or less of compressed or uncompressed, binary or non-binary item data.

Using this property to push item data, whether you’re pushing a single item or batches of items, involves several steps which are detailed in this article.

When pushing a single item

When pushing a single item, you must first create a file container. You then upload the item binary data into the file container. Finally, you perform an Add or update a single item call using the compressedBinaryDataFileId property to reference the file container in which the item data is stored.

Step 1: Create a file container

Perform a POST /files HTTP request to create a file container. Note the uploadUri and fileId from the call response.

Step 2: Upload the item binary data into the file container

Perform the following PUT uploadUri request to upload your item data into the file container you created in step 1.

Request template

Request definition

PUT <MyUploadURI> HTTP/1.1
​
<HTTPHeaders>
​
<MyItemData>

Replace:

  • <MyUploadURI> with the uploadUri you got from step 1.

  • <HTTPHeaders> with the requiredHeaders you got from step 1.

  • <MyItemData> with your uncompressed or Deflate, GZip, LZMA, or ZLib compressed item data. Your request body must be in binary format.

Important

Don’t Base64 encode your item data.

Sample request

The following example shows how to upload Deflate compressed item data.

Request definition

PUT link:https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/mycoveocloudv2organizationg8tp8wu3/05ee7647-15a6-464a-8ffd-cd2fb6c1a03b[...] HTTP/1.1
​
x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream

Request body

eJyzySjJzbHjUgACm4zUxBQoUx+JnZSfUglhQlQZ2rlWJOYW5KQCVRkiSRTYhWRkFisAUWKeQipEiZ6NfoEdF6eNPsQQoA6QdQDMhB8E

Successful response - 200 OK

{}

Step 3: Use the fileId as the CompressedBinaryDataFileId value

Once the request you performed in step 2 has successfully returned, you can use the fileId you got from step 1 as the compressedBinaryDataFileId property value.

The following example shows how you can use the compressedBinaryDataFileId property to specify item data when performing a single push operation (see Add or update a single item in a Push source).

Important

When using the compressedBinaryDataFileId property to push a single item, your query string should include the compressionType parameter along with the compression algorithm that was applied on the data:

  • UNCOMPRESSED

  • DEFLATE

  • GZIP

  • LZMA

  • ZLIB

The default value is ZLIB.

Sample request

Request definition

PUT https://api.cloud.coveo.com/push/v1/organizations/mycoveocloudv2organizationg8tp8wu3/sources/mycoveocloudv2organization-rp5rxzbdz753uhndklv2ztkfgy/documents?documentId=file://example.html&compressionType=DEFLATE HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer **********-****-****-****-************

Request body

{
  // ...Item metadata...
  "compressedBinaryDataFileId": "b5e8767e-8f0d-4a89-9095-1127915c89c7",
  "fileExtension": ".html",
  // ...Item permission model (if applicable)...
}

Successful response - 202 Accepted

{}

When pushing batches of items

Using the compressedBinaryDataFileId property to upload a batch of items involves creating file containers for each item, and another for the batch of items. You then upload the binary data of each item into its file container. The next step is to upload the batch of items into the batch file container, using the compressedBinaryDataFileId property to reference each item’s file container.

Using compressedBinaryDataFileId in a batch update | Coveo

Finally, you can push the batch file container to your source.

Step 1: Create the file containers

For each item to be added or updated, use the POST /files HTTP request to create a new Amazon S3 file container. Note the uploadUri and fileId of each file container you create.

Similarly, create a file container for the batch of items you want to add or update. Note its uploadUri and fileId as well.

Step 2: Upload the binary data of each item into its file container

For each item to be added or updated, use the PUT uploadUri HTTP request to upload the item data into its file container.

Request template

Request definition

PUT <MyUploadURI> HTTP/1.1
​
<HTTPHeaders>
​
<MyItemData>

Replace:

  • <MyUploadURI> with the uploadUri you got from step 1.

  • <HTTPHeaders> with the requiredHeaders you got from step 1.

  • <MyItemData> with your uncompressed or Deflate, GZip, LZMA, or ZLib compressed item data. Your request body must be in binary format.

Important

Don’t Base64 encode your item data.

Sample request

The following example shows how to upload Deflate compressed item data.

Request definition

PUT link:https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/mycoveocloudv2organizationg8tp8wu3/05ee7647-15a6-464a-8ffd-cd2fb6c1a03b[...] HTTP/1.1
​
x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream

Request body

eJyzySjJzbHjUgACm4zUxBQoUx+JnZSfUglhQlQZ2rlWJOYW5KQCVRkiSRTYhWRkFisAUWKeQipEiZ6NfoEdF6eNPsQQoA6QdQDMhB8E

Successful response - 200 OK

{}

Step 3: Upload the batch of items using the compressedBinaryDataFileId property

Upload your items into your batch file container using the PUT uploadUri HTTP request.

In the addOrUpdate child items, use the compressedBinaryDataFileId property to reference the fileId of the file container into which you uploaded the item’s binary data.

Important

When using the compressedBinaryDataFileId property to push the data of an item in a batch, the document body of this item should include the compressionType property along with the compression algorithm that was applied on the data:

  • "UNCOMPRESSED"

  • "DEFLATE"

  • "GZIP"

  • "LZMA"

  • "ZLIB"

The default value is "ZLIB".

Sample request

The following example shows how to upload a batch of items to a file container, referring to an uploaded compressedBinaryDataFileId in an item.

Request definition

PUT link:https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/mycoveocloudv2organizationg8tp8wu3/8d3jf7sh-8d3h-sd89-d1s2-d9fj2ls9dnv7[...] HTTP/1.1
​
x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream

Request body

{
  "addOrUpdate": [
    {
      "documentId": "file://example1.html",
      "compressedBinaryDataFileId": "b5e8767e-8f0d-4a89-9095-1127915c89c7",
      "compressionType": "DEFLATE",
      "fileExtension": ".html",
      // ...Item metadata...
      // ...Item permission model (if applicable)...
    },
    {
      "documentId": "file://example2.html",
      "compressedBinaryDataFileId": "694de324-a895-497c-9624-c14a9424a6ac",
      "compressionType": "DEFLATE",
      "fileExtension": ".html",
      // ...Item metadata...
      // ...Item permission model (if applicable)...
    },
    // ...More items to add or update...
  ],
  "delete": [
    // ...Items to delete...
  ]
}

Successful response - 200 OK

{}

Step 4: Push the file container into your push source

Push the file container in which you just uploaded your batch of items into your source using the PUT /sources/{sourceId}/documents/batch HTTP request.