Simulate FIFO Queue with Amazon DynamoDB and NodeJs

This could be just a concept and has worked for me, which does not imply that it will work for all, and it may even have some serious blunder hidden under which may be found later on during run time.

From a supplier we are getting inventory which is of one time use only. Supplier will provide the inventory only on request, and we do not wish to see the inventory is exhausted. Also we wanted to make sure each bach gets finished in the sequence of applying the same. When we were in MySQL / MariaDB this would have been just another table sorted on primary key and a flag status = ENUM(“new”,”used”). But we are in the “serverless” hype and on AWS with DynamoDB.

Since our main project was having a table already deployed, with a STRING partition key (parkey) and STRING sort key (itemtype), I was particular to stick to this concept.

One row with parkey = “store-keeper” and itemtype = “store-keeper” controlled the queue position, and configuration, further fields were there and the final json is
{
"parkey": "store-keeper",
"itemtype": "store-keeper",
"headAt": 1,
"tailAt": 1,
"minFree": 20
}

To explain things, when we know the exact partition key and sort key, manipulation of row in DynamoDB is very easy^1. When we update stock by adding more inventory, the tailAt is incremented with a request for returning the UPDATED_OLD, will be used in the parkey value for itemtype store-inventory and property inventory will contain the “supplier single use item”. When we consume, the headAt is incremented with a request for returning the UPDATED_OLD will be used in the parkey value for itemtype store-inventory for issuing a DynamoDB get-item api call². Subsequently will be updated using DynamoDB put-item with the added property of “consumed_by”. The inventory structure follows

{
"parkey": "store-inventory: < id > ",
"itemtype": "store-inventory",
"inventory": "our supplier single use item"
}
As soon as the consumption is done, an Inventory Check can be done and if reached minFree level or lower, send intimation to the supplier. For the same we used AWS SES send-templated-email option³. Create your templates using your favourite text (json) editor and update them to the SES console using aws cli⁴.

Though we referred to DynamoDB and its direct documentation, the code uses a complimentary wrapper which includes the marshaller and unmarshaller. The wrapper is also part of the AWS SDK, which is DynamoDB.DocumentClient

Code – NodeJS module

Link: https://s3.amazonaws.com/documentsandmedia/dynamodb-queue.zip

1. https://aws.amazon.com/blogs/database/choosing-the-right-dynamodb-partition-key/
2. https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_GetItem.html
3. https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-personalized-email-api.html
4. https://docs.aws.amazon.com/sdk-for-php/v3/developer-guide/ses-template.html