Automatic shipment completion at Fr. Meyer’s Sohn

About Fr. Meyer’s Sohn

Fr. Meyer’s Sohn – or FMS for short – is a long-established global freight forwarding and logistics company that has been around since 1897.

The family-owned company specialises in the transport of bulk goods such as forestry and agricultural products as well as steel. It offers solutions for sea, air and truck transport as well as for rail and inland.

Over 850,000 TEU (standard containers) are transported annually. In the world of sea freight, they are among the 10 largest shipping companies. They are the market leader in the transport of paper and pulp.

Challenge

FMS organises the transport of containers that are shipped all over the world. Each of these shipments is recorded in an internal system (SeaStep).

The process begins with the client sending an e-mail to FMS. A clerk transfers the information from the e-mail into a shipment mask, which is the destination of the shipment, type and weight and volume of the cargo, as well as desired departure and arrival times. Further information on the operation, such as suitable transport routes, ports of departure and destination, etc., must be added.

This information must be re-entered each time for each shipment. Due to thousands of annual shipments, this process is constantly repeated. There is therefore great potential for savings in automating the process.

Since the transport routes are often similar, it is reasonable to assume that the information is predictable based on a few given pieces of data, such as the ordering party, starting port, shipping destination or type of goods. This data is usually mentioned in the email correspondence about a shipment.

So if it were possible to read the data from the email correspondence and then predict the other fields on the basis of that, the shipment masks could be pre-filled automatically.

The object of our project at FMS was to develop a procedure that is able to automatically fill in missing information correctly.

Solution

A procedure based on Named Entity Recognition is used to extract the data from the emails. For example, ports, container quantities, date values and names of contact persons are recognised.

Any missing information is predicted based on historical shipment data.

In our project with FMS, we mainly focused on the second step:

Automatic shipment completion based on historical data.

The mask with the fields pre-filled from the emails still contains many missing fields that need to be completed.

To solve this problem, hundreds of thousands of historical consignment masks are available that are completely filled in. The task is predestined for predicting the missing fields with machine learning techniques. On the one hand, there is a large data set that can be used to teach supervised learning methods, and on the other hand, there are obvious patterns in the data. For example, some fields correlate strongly with each other; customers often choose similar routes with similar loads. Some values also inevitably result from others (e.g. port of loading Hamburg → shipped via Germany). These “business rules” must be derived automatically from the data.

With classical machine learning methods such as classification or regression, individual values are predicted. The challenge in our case is that many different values have to be predicted simultaneously. For this class of problems, so-called imputation methods are used.

We first evaluated a state-of-theart imputation algorithm (MissForest), but it turned out to be too inaccurate and unperformant for this use case. Therefore, we developed an alternative approach based on the K-Nearest-Neighbours algorithm and a ranking method (BM25). Elasticsearch search seemed to be the tool of choice for this.

Result

By searching the historical data and then aggregating over the most relevant search hits, the most likely values for the missing digits in the consignment masks can be identified. We were able to optimise the process so that even if almost all values of a consignment are missing, they are correctly predicted with an accuracy of over 90%. In addition, the system is highly performant, so that a shipment mask is filled in real time, based on all currently available data.

Patrick Rosendahl
Head of IT at FMS

We had the pleasure of working with exdatis on a data science project and they were a great team. They have a wealth of knowledge and experience in this field and were able to execute our project flawlessly from concept to completion.