Understanding EDI

Understanding EDI EDI vs. ETL vs. API Why EDI matters Use cases Basic overview Inbound Transactions Outbound Transactions Supported transactions Inbound transactions Outbound transactions Common EDI setup To VAN or not to VAN What is a VAN? How does a VAN work? What is the value of using a VAN? How does VAN pricing work? Why would a company opt out of using a VAN? What to expect in Implementation What to expect in daily life ETL

EDI, or Electronic Data Interchange, can be widely used in DEACOM to automate processes, send notifications based on certain events, and monitor system activity, among other things. This page details the importance of EDI functionality, various use cases that can benefit most companies, a basic overview of the supported inbound and outbound transactions, and expectations when implementing EDI and working with it in daily life.

EDI vs. ETL vs. API

To understand why EDI matters and how it can be used to improve business operations, it is helpful to understand what it is in comparison to its existing counterparts. The three players in this game are EDI (Electronic Data Interchange), ETL (“Extract, Transform, Load”), and API (Application Programming Interface).

First, we’ll dive into EDI. Companies use EDI to digitally send information between systems in a standardized format. This digital information replaces the need for paper documents or faxes. EDI transactions can be used for a multitude of processes (with heavy uses including order entry, shipment, and invoicing) and is most effective when a company wants to automate processes to save time and money.

EDI transactions can be in either X12 or XML format. Both contain the same information, just formatted in different ways. These transactions are transmitted between the invested parties, referred to as trading partners, which can either opt to use the basic format or require the inclusion of several company- or industry-specific variables. The requirements depend on the companies included in the transaction and are generally dictated by the larger company. Think of doing business with a massive company, like Walmart, compared to a medium-sized company. The Walmart will have their own requirements for each type of transaction based on their standard processes. Therefore, companies that wish to do business with Walmart must comply with their requirements or risk order rejection.

EDI is an asynchronous form of communication, meaning there is not an immediate response between trading partners; It is more like an email or fax. Originating in the 1960’s, EDI transactions are labeled in North America as 3-digit numbers. This format is slightly different from other countries, which use EDIFACT or other similar standards.

Second comes ETL, which is a type of data integration including three main steps (thus the “Extract, Transform, Load”). The ETL ‘engine’ reads a file, analyzes and locates the necessary data, extracts it from the file, then loads the data into the appropriate areas of the receiving system, i.e. loading the contents of an Excel file.

ETL is a more generic term for pulling and pushing data files from one system to another. All EDI is considered to be ETL, however not all ETL is EDI. DEACOM uses both EDI and ETL methods of data transfer and has developed the ‘EDI tools’ that are available in DEACOM so that they can be repurposed for ETL needs. An example of how DEACOM uses the ETL method is our data conversion process. More information on ETL and useful scenarios are contained later in this booklet.

Third and final is API, a generic term with a more modern, synchronous way of systems talking. While DEACOM does use APIs in other areas of the system (taxes, shipping with FedEx and UPS, eCommerce), the EDI world in general has no plans to adopt API methods, and therefore EDI standards will remain.

Why EDI matters

There are four main reasons why EDI matters. Regardless of the business or industry, EDI can be used to drastically increase data accuracy and efficiency, easily maintain compliance, and scale operations without missing a beat. Essentially, it can be leveraged to bridge the gap between an increasing number of orders and a fixed allotment of personnel or time. Each of these reasons is further detailed below:

• Accuracy. EDI transactions follow rules rather than keystrokes and tout repeated standards with a high level of accuracy. For a typical company, EDI can be used to process 3,000 orders in a one-week span while encountering only 15 errors or less, equating to a 99.5% accuracy rate – far higher than would occur with an entirely manual process. By eliminating manual processes, order errors become almost non-existent.
• Efficiency. Hand in hand with accuracy is the efficiency of utilizing EDI. The time it would take to manually enter thousands of orders is severely dwarfed when EDI is brought into the mix. An import profile has the capacity to process dozens of orders in the time it would take one to be manually created.
• Compliance. Large companies or trading partners, such as Walmart and Target, that use EDI transactions typically have their own set of specific requirements. Configuring the necessary maps to handle these requirements ensures your company is in compliance with the set standards for every order and allows seamless communication between both parties.
• Scalability. Companies utilizing EDI can easily and massively increase productivity without increasing costs or labor hours. This is because once the EDI maps are configured, all transactions fire automatically, requiring minimal to no manual intervention. It becomes much easier to process orders, monitor inventory, and send invoices without worrying about manually touching every step of the order fulfillment process. Companies can leverage EDI to scale their operations – to jump from 50 customers to 500 – without the need to supplement their data entry staff.

Use cases

We can talk all day about how EDI is a top-notch solution, but the evidence truly lies in the numbers. The following are recent, real-life examples of how our customers’ utilization of EDI has been able to ramp up data entry virtually error-free.

• 2,115 Sales Orders imported in 290 minutes. For this customer, that is almost a month’s worth of orders and equates to 7.29 orders/minute with a .003% failure rate.
• 957 Sales Orders imported in 124 minutes. For this customer, that’s a week’s worth of orders and equates to 7.72 orders/minute, again with a .003% failure rate.
• 173 Sales Orders imported in 22 minutes. For this customer, that is one day’s worth of orders and equates to 7.86 orders/minute, this time with 0% failure.

We invite you to compare those numbers with your own order entry and error rates. Can you match that with your current processes? Realistically, it would take a minimum of one to two full time employees to match the entry rates, but error rates of near zero is unheard of when it comes to manual data entry. Still not convinced? Keep reading for more information on how implementing and fully utilizing EDI can make a world of difference at your company.

Basic overview

EDI transactions can be either inbound (ex: your company receives an order from one of its customers) or outbound (ex: your company submits an order to one of your suppliers), both of which are discussed below.

Inbound Transactions

For inbound transactions, one partner (usually the customer) creates the original file and transfers it to a local folder that both partners have access to. This folder may be through a Value-added network or VAN, such as SPS Commerce, or it may be direct, but it is almost always done using SFTP. Once the file is shared by the first partner (the customer), it shows up on the second partner’s (the selling company’s) server.

The EDI logic is constantly looking for files like this and so once it recognizes a new file has been loaded, it picks the file up, parses through the map, and makes the appropriate updates in DEACOM. Upon receipt of the file, a standard practice is for the receiving partner to send a Functional Acknowledgement (997) to the first partner to indicate that the file was indeed received. This 997, discussed more in depth later in this booklet, does not indicate that all information on the file was accurate; it simply lets the customer know that their order was received by the seller.

Below are examples of an 850 transaction, on the left in X12 format and on the right in XML format.

Outbound Transactions

Outbound transactions occur due to a specific event occurring in the system or a timed event firing. This event, whether it be a Purchase Order being saved or a shipment being processed, generates the EDI file based on the maps configured and sends the file to the receiving partner. Just like inbound transactions, submission of outbound files is usually done via SFTP and more often than not using a VAN. To confirm that the outbound transaction was sent, a 997 may be received or the VAN, if being used, will handle reconciliation to ensure all files were sent successfully.

Supported transactions

DEACOM supports inbound and outbound transactions, both of which can be broken down into common and advanced uses.

Inbound transactions

For inbound transactions, the most commonly used are:

• Sales or Purchase Order (850) / Grocery Sales or Purchase Order (875)
  • The 850 is used to place an order and generally includes items, prices, quantities ordered, shipping details, payment terms, any applicable discounts, etc.
  • Can be used for both Sales and Purchase Orders, since the required information is essentially the same. It simply requires some different tags to trigger a sale versus purchase transaction in the receiving system.
  • The 875 is essentially the same as the 850 but used specifically for grocery products.
  • The 875 transaction usually includes the same data as 850 transactions plus any other information unique to the grocery industry necessary for order fulfillment.
• Functional Acknowledgement (997)
  • Sent as a response to other EDI transactions received.
  • Serves as a confirmation of EDI receipt; however it does not confirm or deny that the trading partner has received all of the necessary information or that the information it has received is correct.
  • More advanced inbound transactions include:
• Warehouse Shipping Order (940)
  • Used by sellers to authorize and instruct a warehouse to ship materials. It may also be used to confirm, modify, or cancel a shipment.
  • Instructs the warehouse on what to ship, how much, and when and includes data such as delivery date, bill-to and ship-to details, and shipping method.
• Order Change Request (860)
  • Used to amend an order after it has been submitted to the supplier.
  • Includes original order information as well as the requested changes, such as adding items to the order, changing quantity or price, or rescheduling the order.
• Carrier Charges (210)
  • Can be thought of as a PO for freight.
  • Provides itemized details of the charges, invoice date and number, shipper and consignee names and information, delivery information, etc.
• Warehouse Transfer Receipt (944)
  • An acknowledgement sent to a manufacturer that its shipment has been received by the warehouse.
  • Typically used by 3PL providers.
• Warehouse Shipment (945)
  • Used by warehouses to provide confirmation to a trading partner that a shipment was made.
  • Includes shipment identification information, ship from and ship-to details, ship date and method, items and quantities shipped, etc.
  • Generally sent as a result of receiving a 940 transaction

Outbound transactions

For outbound transactions, the most commonly used are:

• Purchase Order (850)
  • Sent from DEACOM to the indicated vendor.
  • Triggered upon the initial creation of a Purchase Order in DEACOM.
• Purchase Order Acknowledgement (855)
  • Used by the seller to confirm the receipt of a Purchase Order.
  • Indicates if the order is accepted, rejected, or accepted with changes (these changes can be freight terms, changing or rejecting line items, etc.).
• Advance Ship Notice (856)
  • Sent in advance of a shipment arriving at its destination.
  • Triggered upon the shipment of a Sales Order in DEACOM.
  • Details the contents of the shipment, order and carrier information, etc.
  • Can be very complicated to implement as each trading partner can have very different requirements.
• Invoice (810) / Grocery Invoice (880)
  • The 810 is the electronic version of an invoice, typically sent in response to an 850 Purchase Order as a request for payment.
  • Generally includes invoice number and date, payment terms, item prices and quantities, discounts, etc.
  • The 880 is essentially the same as the 810 but used specifically for grocery products.
  • The 880 transaction usually includes the same data as 810 transactions plus any other information unique to the grocery industry necessary for invoice processing and payment.
• Functional Acknowledgement (997)
  • Sent as a response to other EDI transactions received.
  • Serves as a confirmation of EDI receipt; however it does not confirm or deny that the trading partner has received all of the necessary information or that the information it has received is correct.
  • More advanced outbound transactions include:
• Order Change Request (860)
  • Used to amend an order after it has been submitted to the supplier.
  • Includes original order information as well as the requested changes, such as adding items to the order, changing the quantity or price, or rescheduling the order.
• PO Change Acknowledgement (855c)
  • Used when a Purchase Order that was previously accepted must be changed (this could be a result of something like not producing enough units to fulfill the original order).
• Warehouse Shipping Order (940)
  • Used by sellers to authorize and instruct a warehouse to ship materials. It may also be used to confirm, modify, or cancel a shipment.
  • Instructs the warehouse on what to ship, how much, and when and includes data such as delivery date, bill-to and ship-to details, and shipping method.
• Warehouse Transfer Shipment (943)
  • An ASN sent to a warehouse (specifically a third-party warehouse) to alert them that a shipment has been sent from the manufacturer.
  • May also be used by manufacturers to authorize a warehouse to accept a customer return.
• Warehouse Shipment (945)
  • Used by warehouses to provide confirmation to a trading partner that a shipment was made.
  • Includes shipment identification information, ship from and ship-to details, ship date and method, items and quantities shipped, etc.
  • Generally sent as a result of receiving a 940 transaction.
• Warehouse Inventory Adjustment (947)
  • Informs a warehouse of inventory quantity or status changes.
• Inventory Status (846)
  • Communicates inventory status information between manufacturers, suppliers,and resellers.
  • Includes relevant information such as quantity on hand, committed and on order quantity, quantity in transit, and backordered quantity.
  • Commonly used to:
    • Provide available inventory levels to potential customers.
    • Notify parties of on hand inventory in various holding locations (warehouses, stores, distribution centers, etc.).
    • Advise customers of overstocked inventory being offered at a discount

Common EDI setup

As seen in the Basic Overview section, EDI transactions are commonly configured in either X12 or XML formats. In addition, it is usually easiest to employ a VAN to handle communications, however files may also be sent directly. Regardless of how the files are formatted and sent, a typical process is as follows:

1. Customer submits their order using an inbound 850 Purchase Order.
2. DEACOM responds using an outbound 855 Purchase Order Acknowledgement.
3. DEACOM creates a Sales Order using the information found in the received 850 file.

Assuming all necessary information was included and accurate in the 850 file, the Sales Order is fulfilled and shipped per standard company processes.

4. The shipping transaction in DEACOM triggers the system to send an outbound 856 Advance Ship Notice.
5. The Sales Order is invoiced per standard company process.
6. DEACOM sends the customer an outbound 810 Invoice.

And that’s it! It’s easy to see how streamlined the order fulfillment process can be by employing the use of EDI. Taking inbound EDI transactions and following standard company processes within DEACOM easily ensures that all necessary data is captured, and all pertinent information is sent to the customer. Leveraging EDI to automatically acknowledge and create orders then send invoices eliminates the need for additional employees who may focus solely on order entry, which can add up to be a huge expense depending on the volume of orders.

To VAN or not to VAN

Once it is decided that a company will be using EDI, one of the first decisions is to choose whether to use a VAN. There are pros and cons to VANs, but before getting into those, let’s define what a VAN is and how it works.

What is a VAN?

A VAN, which stands for Value-added network, is a network provider hired by a company to facilitate EDI services and secure communications between trading partners (your company and its customers). To draw an easy comparison, VANs can be thought of as mail carriers; they take incoming items (letters, packages) and direct them to the proper recipient (P.O. boxes, businesses, personal residences) using various methods (ground or air, next day or overnight).

In today’s market, there are around twelve companies that act as VANs. DEACOM has successfully worked with VANs such as SPS Commerce, Digital Movers, Covalent Works, Sterling Commerce, and GXS to name a few. Companies like yours hire VANs to act as intermediaries between the DEACOM system and their customers.

How does a VAN work?

As previously stated, the VAN is an intermediary between a company and its’ trading partners. All communications (orders, acknowledgements, shipping and receiving notifications, invoices, etc.) between the two groups are sent directly to the VAN. These communications can come in many ways and forms, so the VAN takes the information received and formats it in a standardized way so that the receiving partner can understand it. The diagram below is a simplified illustration of how this relationship typically exists.

What is the value of using a VAN?

The VAN can handle translations between partners, depending on each of their requirements. Translations can be from grocery to non-grocery, EDI to EDIFACT, XML to X12, etc. Essentially, the VAN smooths out communications between the two partners to ensure each is getting the necessary information in the format they can handle. Bonus: these translations can also apply to languages, where one company ‘speaks’ to the VAN in English and the VAN translates the information into German, Japanese, French, etc.

VANs also typically handle 997 reconciliations, meaning they take a list of all orders received and compare it to all orders fulfilled. When it finds that one or more orders received have not yet been fulfilled, it can let the company know that an order may have been missed due to something like a poor connection.

A key value proposition of VANs lies in the exponentially easier onboarding of new partners. While initial setup with a VAN make take some time, the benefit is that it takes essentially no time to add a new partner. This easily allows a company to scale from 50 customers to 500 without having to mirror the scale in their workforce or worry about how to support long-term growth.

Additionally, VANs can be used to help companies maintain trading partner compliance. Some partners have their own rules for receiving EDI transactions and reject any that don’t tick each of their check boxes. VANs can ensure all transactions generated comply with these special conditions, which could be as simple as including a few additional fields, have a carbon copy on the file, or mandating the use of AS2 (a very highly secured connection used instead of SFTP).

How does VAN pricing work?

Typically, the price of using a VAN is based on volume, mostly measured by transaction, sometimes measured by character. VANs commonly require long-term contracts with monthly volume limits but do occasionally permit overages (similar to how a cell phone plan may work). This contract requirement and pricing structure may be one reason that a company chooses to avoid using a VAN, which leads us to the next point.

Why would a company opt out of using a VAN?

There are several reasons, the most common being:

• Cost and/or contract requirement – Depending on the VAN’s pricing structure, the company may be averse to signing a contract or paying per character. VANs are typically not the best choice for a company with razor thin margins.
• Multiple contacts – One of the best perks of using a VAN is the way it handles and translates the information. That being said, when an error does occur, it may require contacting multiple companies within the VAN network to get resolution. The VAN may claim the issue lies on the trading partner side, forcing your company to reach out to the partner directly. Depending on the number of trading partners in the network, this can be frustrating as it requires the company to maintain multiple relationships.
• Unnecessary services – Another perk of using VANs can also translate into a con depending on the functionality of a company’s ERP or the scale of their operations. Some VANs offer additional services (label generation, management reporting), which don’t necessarily hold value for every company. These extra services can be rendered unnecessary when, for instance, there are only a couple trading partners. In that case, direct communication may be easier and cheaper to maintain.

What to expect in Implementation

EDI can be implemented at any time, either during the initial DEACOM implementation process or after a company has been using the software for several years. It is up to the company, their requirements, and their processes.

Throughout the EDI implementation process, Deacom’s primary role is to build and validate the maps that will be used. Because of the nuanced nature of some maps, it is necessary to have several good sample files from each of the trading partners that will be using the EDI connection. This allows the Deacom representative to ensure all data is getting captured and communicated.

To begin the implementation of EDI, an initial kickoff call will be held to get all involved on the same page and to walk through a questionnaire regarding how EDI is used at the company today and the related business practices.

After the call, EDI-specific user fields and calculations will be built and those, along with the basic maps, implemented into a Deacom local system. A Deacom representative will then work through common transactions in the system to fit use cases, get a feel for the business operations, and troubleshoot where things go awry.

Based on the basic maps installed, the needs of the business, and troubleshooting results, maps and/or data is changed where needed – everything goes through the process of test, analyze, iterate.

Once success on all fronts is established on the local system, the fields, calculations, and maps are copied to the customer system and re-iterated. Finally, a Deacom representative teaches the process and you’re off!

What to expect in daily life

When every aspect is working right, you shouldn’t have to think about your EDI transactions. The only real activity that may need to occur is a periodic review of EDI health using the Trigger and EDI Import history reports inherent in DEACOM. These reports provide a history of the transactions that have been attempted, including the date and time of the event, if it was a success or failure, and if applicable, what error was encountered. The employee(s) responsible for monitoring EDI health can then use the results to investigate any failures and address the root cause.

Another aspect that may warrant attention is when either a trading partner, your business practices, or DEACOM changes. Partners may periodically choose to add new requirements or adjust their accepted formats. Deacom and potentially the VAN can help implement these changes to minimize the impact on operations.

On the other hand, your company may decide to make changes to their business model or data structure, or choose to update to a newer version of DEACOM. On the topic of business and data structure changes, once they are solidified, it is imperative to update all EDI maps to match these changes so that things continue to run smoothly and require monitoring versus maintenance. As for software updates, it is very rare that any change would have an impact on EDI operations. The most important thing is to ensure all EDI transactions and processes are tested in the test environment and confirmed as functioning properly before applying the update to the production environment.

ETL

While the bulk of this page has discussed the aspects of and processes for EDI, ETL can also be used for a slightly wider range of applications and yield great benefits. To recap, ETL is the process of extracting data from one or more sources, transforming it into a different format, then loading it into the receiving system. Typical examples of how ETL is used in DEACOM include loading the contents of an Excel file and our data conversion process.

In addition, there are several more involved applications for using ETL to further your business. ETL can be used to handle incoming data from powerful tools such as websites and point of sale systems, to track information for items like totes and pallets, and in conjunction with timed triggers configured in DEACOM. Let’s dive into these more involved examples.

• Service charges and storage fees. ETL can best be utilized to create Sales Orders for inventory holding charges. For example, your company creates a Purchase Order to receive material into a warehouse. That receipt of inventory fires an EDI-type trigger that creates a file, containing the item and quantity details, that then gets dropped into a network folder. Meanwhile, an EDI import is polling that network folder every x minutes so that when a file gets dropped, the EDI import processes the file and creates a Sales Order to the customer for the appropriate storage fee. No more worries about tracking and invoicing for storage costs!
• Websites and POS systems. Websites are an excellent way to further your customer base and supplement sales. But just as important as designing a beautiful, functional website is handling the orders that are placed as a result. Rather than having one or more employees responsible for collecting, inputting, and reconciling incoming orders, ETL can be used to take all information from the order and create a Sales Order within the DEACOM system. Once created, the order fulfillment process is the same as a manually entered order. Plus, the same health reports mentioned previously can be used to monitor this activity and resolve any potential errors. The result: reaching more customers to receive more orders without having to shell out for more payroll.
• Timed triggers. Data import profiles can easily be created then set to run on a schedule from within DEACOM. Once the schedule is saved, the import profile updates a field in the database which then fires a trigger. A common use for this option is implementing the positive pay process, in which a text file is created to send to the bank after a check run has occurred.