Overview of Spark v1 (the current version)

Database of eligible deals

• Updated manually once per week
• Verified StorageMarket deals only
• Eligible deal is (minerId, payloadCid, clientId) , built as follows:

{
  minerId: DealProposal.Provider // f0...
  payloadCid: DealProposal.Label // bafy...
  clientId: DealProposal.Client  // f0...
}

Retrieval check (v1)

1. Query blockchain state (Filecoin.StateMinerInfo(minerId, null)) to obtain miner’s peer ID
2. Query IPNI at https://cid.contact to find the advertised retrieval provider for payloadCid that matches the miner’s peer ID
3. Fetch payloadCid using the advertised provider address and retrieval protocol (HTTP or Graphsync)

Spark v2 design (the upcoming version)

Database of eligible deals

• Updated in near-real-time by observing actor events

• All verified deals (StorageMarket, DDO, potentially other deal types added in the future)

• An eligible deal is (minerId, pieceCid, pieceSize, clientId) built as follows using the claim metadata (docs):

  {
  	minerId: metadata.provider      // 1660795 means f01660795
  	pieceCid: metadata.piece-cid    // baga...
  	pieceSize: metadata.piece-size  // 34359738368
  	clientId: metadata.client       // 2147046 means f02147046
  }
  

• Additionally, we need to keep track of the lifetime of the sector and claim to understand the start and end of the time window in which the deal is “active” and expected to be retrievable. All information should be available in the actor events, but we need to research how exactly to determine the start and end.

Retrieval check (v2)

1. Query blockchain state (Filecoin.StateMinerInfo(minerId, null)) to obtain miner’s peer ID.
2. Build contextID from (pieceCid, pieceSize) using the same algorithm as Curio (source code).