How To Painlessly Renew Legacy Protocols in Logistics: EDIFACT to REST API Case Study

Sometimes companies stick with outdated technologies because they believe replacing them will take too much effort. We created a so-called MVP (a minimum viable product) of a solution to validate it works so that the Client could develop it further after it proves effective.

# The Problem

Our Client, a company, running logistical operations, had come across the problem of legacy protocols. As the most in the field, the Client was using an interactive exchange protocol to perform logistical operations. These protocols are outdated: some of them date back to the 50s and 60s. Nevertheless, they are still in use.

The Client had to conduct all communication with their partners and contractors in an outdated EDIFACT (International standard for electronic data interchange) version that doesn’t scale to cloud. For example, this outdated protocol was holding back the integration of data into a new format. EDIFACT format was inspired by military logistics and traces its roots back to the end of 1940s. Surprisingly it’s still widely used today, in a completely different world.

At Moqod, we built an MVP as a proof-of-concept for a new solution. Further, in this case study, we will show you how.

# The Challenge

Using modern cloud solutions, we had to build a system enabling the Client to convert EDIFACT into a modern format, easier to store and analyze. On top of that, we needed to do it fast, and with limited resources. Ultimately, we focused on solving the business problem, rather than replacing infrastructure. 

Our choice fell on AWS (Amazon Web Services) and open data libraries. 
Alternatively, we would have to:

• Set up a server
• Resolve an authorization and authentication problem
• Provide monitoring
• Provide support and maintenance

Modern cloud solutions enable us to skip these steps and to cut expenses since we pay AWS cloud fee only.

# The Choice Of The Solution

Instead of building a converter from scratch, we chose to use one of the open-source libraries for converting EDI into JSON. Upon considering a number of options, we decided to use the EDI-JSON project. The use of an open-source library has a number of limitations, but they didn’t interfere with our task.

Further, we needed to pick one of the options for building the service:

1. Build a service using Java, ‘wrap’ a Web-server with this service into a Docker container and launch it on a virtual server
2. Choose a Serverless model and deploy a service on a Cloud

The cloud environment provided out-of-the-box scalability, availability, and authentication, so we considered option 2 to be the best for our requirements. All we had to do is write a request processing code, and deploy this solution into a cloud infrastructure.

We chose an AWS cloud infrastructure (a combination of Amazon API Gateway and AWS Lambda). We used a free open-source version, although we can further switch to the paid version if we need to scale up.

# The Description of The Solution

As we chose the combination of Amazon API Gateway and AWS Lambda, our task was to write Lambda functions for processing requests.

Writing Lambda functionsThe EDI-JSON project already contains a pre-written EdiToJsonDriver, which enables us to convert an EDI file into a JSON file. But because our task was to write a Lambda function, we needed a module that would operate with strings instead of files. Therefore, we used EdiToJson class (compiled class is contained in a edireader-json-basic-5.5.19.jar archive, which is used in EdiToJsonDriver).

Further, we wrote an EdiToJsonConverter implementing a RequestHandler interface with handleRequest method, which receives a JSON-object with a field “data” (a string with data in EDI format) and returns a JSON-object received during converting EDI into JSON. These are the instructions we used to write a Lambda function: AWS Lambda function handler in Java

Assembling a zip-archive for deployment
Zip-archive of Lambda functions for deployment was created according to this manual.

The created zip archive can be uploaded into AWS Lambda through an AWS console.

How to upload zip archive into AWS Lambda through an AWS console.

API Gateway Creation and Setup
Using the AWS documentation, we created REST API with POST-method /edifact-json, and with previously created Lambda function for processing requests.

Creating REST API with POST-method /edifact-json

We chose a mechanism with an API Key for authentication.

API Key for authentication

Creating a Usage Plan and an API KeyIn order to use an API Key in REST API, we need to create a Usage Plan and API Key according to instructions.

Query example:

Query(using cURL)

curl -X POST \
https://.execute-api.eu-central-1.amazonaws.com/testing/edifact-json/ \
-H 'Content-Type: application/json' \
-H 'cache-control: no-cache' \
-H 'x-api-key: **' \
-d '{
"data": "UNB+UNOA:1+01010000253001+O0013000093SCHA-Z59+991006:1902+PAYO0012101221'\''\nUNH+1+INVOIC:D:97A:UN'\''\nBGM+381+1060113800026+9'\''\nDTM+137:199910060000:102'\''\nNAD+BT+VAUXHALL MOTORS LTD::91'\''\nRFF+VA:382324067'\''\nNAD+SU+2002993::92'\''\nRFF+VA:123844750'\''\nCUX+2:EUR'\''\nPAT+1'\''\nDTM+140:19991031:102'\''\nLIN+++090346642:IN'\''\nQTY+12:54:PCE'\''\nMOA+203:1960.29'\''\nPRI+AAA:3630.1724::NTP:100:C62'\''\nRFF+SI:165480'\''\nDTM+11:199909280000:102'\''\nRFF+ON:X18V00003'\''\nRFF+TN:AB1'\''\nTAX+7+VAT+++:::0'\''\nNAD+ST+023::92'\''\nUNS+S'\''\nMOA+77:1960.29'\''\nTAX+7+VAT'\''\nUNT+24+1'\''\nUNZ+1+PAYO0012101221'\''"
}

Response:

{
"interchanges": [
{
"UNB_01_SyntaxIdentifier": {
"UNB_01_01_SyntaxIdentifier": "UNOA",
"UNB_01_02_SyntaxVersionNumber": "1"
},
"UNB_02_InterchangeSender": {
"UNB_02_01_InterchangeSenderIdentification": "01010000253001",
"UNB_02_02_IdentificationCodeQualifier": ""
},
"UNB_03_InterchangeReceiver": {
"UNB_03_01_InterchangeReceiverIdentification": "O0013000093SCHA-Z59",
"UNB_03_02_IdentificationCodeQualifier": ""
},
"UNB_04_DateAndTimeOfPreparation": {
"UNB_04_01_Date": "991006",
"UNB_04_02_Time": "1902"
},
"UNB_05_InterchangeControlReference": "PAYO0012101221",
"transactions": [
{
"UNH_01_MessageReferenceNumber": "1",
"UNH_02_MessageIdentifier": {
"UNH_02_01_MessageType": "INVOIC",
"UNH_02_02_MessageVersionNumber": "D",
"UNH_02_03_MessageReleaseNumber": "97A",
"UNH_02_04_ControllingAgency": "UN"
},
"segments": [
{
"BGM_01": "381",
"BGM_02": "1060113800026",
"BGM_03": "9"
},
{
"DTM_01": {
"DTM_01_01": "137",
"DTM_01_02": "199910060000",
"DTM_01_03": "102"
}
},
{
"NAD_01": "BT",
"NAD_02": {
"NAD_02_01": "VAUXHALL MOTORS LTD",
"NAD_02_03": "91"
}
},
{
"RFF_01": {
"RFF_01_01": "VA",
"RFF_01_02": "382324067"
}
},
{
"NAD_01": "SU",
"NAD_02": {
"NAD_02_01": "2002993",
"NAD_02_03": "92"
}
},
{
"RFF_01": {
"RFF_01_01": "VA",
"RFF_01_02": "123844750"
}
},
{
"CUX_01": {
"CUX_01_01": "2",
"CUX_01_02": "EUR"
}
},
{
"PAT_01": "1"
},
{
"DTM_01": {
"DTM_01_01": "140",
"DTM_01_02": "19991031",
"DTM_01_03": "102"
}
},
{
"LIN_03": {
"LIN_03_01": "090346642",
"LIN_03_02": "IN"
}
},
{
"QTY_01": {
"QTY_01_01": "12",
"QTY_01_02": "54",
"QTY_01_03": "PCE"
}
},
{
"MOA_01": {
"MOA_01_01": "203",
"MOA_01_02": "1960.29"
}
},
{
"PRI_01": {
"PRI_01_01": "AAA",
"PRI_01_02": "3630.1724",
"PRI_01_04": "NTP",
"PRI_01_05": "100",
"PRI_01_06": "C62"
}
},
{
"RFF_01": {
"RFF_01_01": "SI",
"RFF_01_02": "165480"
}
},
{
"DTM_01": {
"DTM_01_01": "11",
"DTM_01_02": "199909280000",
"DTM_01_03": "102"
}
},
{
"RFF_01": {
"RFF_01_01": "ON",
"RFF_01_02": "X18V00003"
}
},
{
"RFF_01": {
"RFF_01_01": "TN",
"RFF_01_02": "AB1"
}
},
{
"TAX_01": "7",
"TAX_02": "VAT",
"TAX_05": {
"TAX_05_04": "0"
}
},
{
"NAD_01": "ST",
"NAD_02": {
"NAD_02_01": "023",
"NAD_02_03": "92"
}
},
{
"UNS_01": "S"
},
{
"MOA_01": {
"MOA_01_01": "77",
"MOA_01_02": "1960.29"
}
},
{
"TAX_01": "7",
"TAX_02": "VAT"
}
]
}
]
}
]
}

# Project Timeline

The solution was implemented relatively fast. It took us up to two days to perform research and one day to implement it.