Later, Dorothy and Emerj’s Head of Research, Daniel Faggella, discuss how Google, AWS, and Microsoft are lowering the barriers to entry for SMEs looking to start in AI and democratizing data science roles that were highly specialized just a few years ago.

Listen to the podcast:

The post Winning buy-in from leadership for AI projects, with CTO Dorothy Li appeared first on Convoy.

Together, they discuss the topic of predicting freight shipments with artificial intelligence. Dorothy explores a logistics use case for AI capabilities in predictive inventory and the impact it is having on manufacturing writ large. Still, there are many challenges, among them that truckers and freight carriers don’t have the equipment (mobile apps, GPS) that other last mile and delivery workers have. They also examine where AI can be applied to help schedule deliveries to avoid unplanned downtime.

Give the podcast episode a listen.

The post Convoy’s CTO Dorothy LI on predicting freight shipments with AI appeared first on Convoy.

This technical challenge is not unique to the freight industry. Economists have long grappled with the real-time identification of turning points in areas such as finance and macroeconomic policy. Their record from decades of research suggests that there are no silver bullets, but also that there is scope for modest technical improvement from current freight industry practices.

In this blog post, we outline how Convoy Science & Analytics built upon research from the intersection of machine learning and empirical macroeconomics to help inform our team’s view of one of the trickiest, most business-critical questions for the freight industry.

The problem: We are all hiking in a fog of uncertainty

Economic forecasting is sometimes likened to hiking in a dense fog: We know in intimate detail the terrain we have traveled, but have only an abstract sense of whether the next step will be into a rock wall or off the edge of a cliff. As time unfolds, there is no definitive answer to whether a single data update is a blip to overlook or the opening salvo of an enduring trend. Head fakes are frustratingly common.

The clear lesson is: While we can speculate about the forces we anticipate will shape markets on the visible horizon, there are no universal laws governing when the pendulum will swing in favor of demand or supply. Peaks and troughs are obvious only in retrospect — in financial markets, in the broader economy, and as we have learned at Convoy, in the trucking market.

But decision makers must decide, even in the presence of pervasive uncertainty — and the stakes in these judgment calls are not trivial. Predicting the direction of the freight market — in particular, spot market prices for trucking services — is essential for pricing the long-term freight contracts that large retailers and manufacturers rely on to move their products. Getting the direction of prices (much less the price itself) wrong can have catastrophic consequences for a contract’s long-term viability.

The solution: A Hidden Markov model of freight business cycles

Generations of macroeconomists have dedicated careers to developing more robust tools to detect economic turning points than the finger-to-the-wind heuristics still common in the freight industry. One such tool is the recession probability calculator for the U.S. economy developed by economists Marcelle Chauvet and Jeremy Piger (see, for example, here and here).

The Chauvet-Piger approach uses a Hidden Markov Model (HMM) — trained on nonfarm employment, industrial production, real personal income, and real manufacturing sales — to identify the odds that the most recent data reflect a durable inflection point in the U.S. economy. (The official arbiter of these turning points is the group of economists who sit on the National Bureau of Economic Research’s Business Cycle Dating Committee.) It is a creative and unconventional application of HMMs, which are widely applied in diverse domains ranging from speech recognition to genomics.

Below, we provide a simple description of how HMMs work, their appeal as a solution for the problem of identifying market turning points in real time, and how we built a HMM for the freight market.

A simple explanation of Hidden Markov Models

Three properties of HMMs make them particularly well-suited to the challenge of identifying market turning points:

First, the core assumptions of HMMs align with economic theory of how market prices move. HMMs model observed price changes as the random output of some underlying probability distribution. Modeling price movements as a random sequence aligns with economic theory (and empirical evidence) of approximately efficient markets across a wide range of industries.

Second, the underlying probability distribution that observed price changes are drawn from is allowed to change over time. Fitting a model to the data requires selecting a finite number of possible underlying distributions (typically called “states”) and then finding the best fit parameters (e.g., mean and variance) associated with each state. For questions about the business cycle, the number of states to select is obvious: There are two, an expansionary state and a recession state.

Third, the model outcome is easily interpreted for non-technical audiences. A key benefit of the HMM for Convoy’s application is that it allows the rigorous quantification of the probability that the most recent sequence of market data signifies a regime change (i.e., that the data shifted from one “state” to another, such as from expansion to recession or vice versa).

A simulated example

Consider the following example, built with simulated data.

In Figure 1, below, the red and white points in the top panel are drawn from one of two Gaussians. The red points are drawn from a Gaussian with a positive mean, which we call state 0, and the white points are drawn from a Gauissian with a slightly negative mean, which we call state 1. If we interpreted these numbers as sequential changes, then the white points (state 1) would be associated with a prolonged decline in the observed data. Points drawn as dots have been correctly classified by the HMM as being in the correct state, while points drawn as x’s have been incorrectly classified.

While there are a few minor misclassifications, the long stretch of white dots is classified correctly for every point when all the model is trained on the full data at once, and with only one misclassification when the model is trained on only the data observed prior to that period. The regime change is detected because the first white dot is negative enough that the probability of the system being in state 1 exceeds 50 percent (shown in the bottom panel). In the middle of the “recession” there are several white points above zero that — even though they are technically more likely to be produced by the state 0 distribution than the state 1 distribution — are nonetheless correctly classified as state 1.

How could this be? A final attribute of the HMM is that it understands that regimes tend to last for a certain amount of time. It captures this by assuming that there is a certain probability of either transitioning between states from one point to the next, or else staying in the same state. In this example we’ve specified that there is a 10% chance of transitioning and a 90% chance of staying in the same state. Therefore, an only slightly positive data point coming after a run of negative points does not offer good enough evidence for a regime change switch. You can see the corresponding probability of being in state 1 dip for these points, but only once dipping below 50% in the sequentially fit data, and even then not getting very close to zero.

Taken together, this makes the HMM an excellent tool for real-time diagnosis of regime switching. In the example above, even the false “recession” classifications don’t have a probability very close to 1. By setting appropriate thresholds, we can make a good recession detector.

Extending the model to the trucking market

Inspired by the work of Chauvet and Piger, we decided to build a HMM to identify turning points in the trucking market.

The first step was to replace the general economic indicators used by Chauvet and Piger with indicators that are more closely tied to the trucking industry and for which a reasonably long time series is available. This is not a trivial choice. While there is extensive theoretical literature (and, in some cases, statutory guidance) about the metrics to track when identifying broader economic fluctuations, the foundations underpinning freight-specific booms and busts stand on much shakier ground.

Our starting point was the intuition — derived from classical Keynesian economic theory — that fluctuations in the freight economy are driven by changes in the aggregate demand to move goods. This led to three categories of training data:

• Models trained on data assuming that turning points are cleanly identified by exogenous demand shocks from sectors of the economy that rely on freight to move goods — for example, metrics such as inflation-adjusted retail sales, factory output, imports, and construction starts.
• Models trained on data that incorporate industry aggregates such as the American Trucking Association’s Truck Tonnage Index and Cass Information Systems’ freight index — though these indexes are typically vulnerable to idiosyncratic methodological limitations and known biases.
• Models that incorporate supply-side metrics — such as heavy truck and trailer production and sales, and trucking industry employment. Incorporating supply-side metrics muddies the Keynesian causal link of strictly demand-side models, but may yield higher accuracy if (as is typically the case) reality is messier than the theory.

The outcome of the best-performing model — which included heavy truck and trailer production and domestic manufacturing output — is shown below. (For all of the models, we transformed the metrics to ensure stationarity and normalized to train on a two-state HMM.) We then used the following two rules to designate high-confidence freight recessions based on the probabilities that the HMM produced:

1. If not currently in a recession, declare one when: Two subsequent months have probabilities exceeding 0.9, or if in any month the recession probability exceeds 0.95.
2. If currently in a recession, declare an expansion when: Two subsequent months have probabilities drop below 0.1, or if in any month the recession probability is below 0.05.

Figure 2, below, illustrates two distinct probabilities. High-probability freight recessions as defined above are indicated by the gray shaded areas.

One is the real-time probability, where at each point in time the model is trained only on data available at that moment. This is what we see in practice as the market unfolds. The second is a “smoothed” probability, which uses all the data available — historical and future — and therefore has higher accuracy and is less noisy.

When we compare these probabilities to spot market prices for trucking services — the ultimate metric that motivated our initial interest in identifying freight market turning points — recessionary periods reliably indicate periods of falling prices, and the end of a recession is always followed by an extended period of rising prices.

The horizon has not been defeated, but it has been quantified

The past half-decade — much less the past two years of pandemic-related supply chain disruptions — have shown us that unexpected economic shocks are not outliers; they are the norm. Determining whether a single month represents a temporary blip or an enduring market reversal has enormous consequences for any business attempting to navigate the freight market. Modern marketplaces require more precision than rules-of-thumb.

By building upon research at the intersection of machine learning and empirical macroeconomics to develop a Hidden Markov Model of freight recession probabilities, Convoy Science & Analytics has been able to help guide better business-critical decisions. There is no single silver bullet and we do not pretend that our probabilities are an all-knowing solution to the devilishly tricky problem of predicting the future. But when considered among a range of other indicators and model outputs, they can help free freight decision makers from the high-stakes guesswork that has historically exacerbated market fragilities.

The post Identifying freight market turning points in real-time with Hidden Markov Models appeared first on Convoy.

Over the past six years, Convoy has built the most tech-connected carrier network in the nation. Today, more than 300,000 trucks move with our GPS-enabled app, which captures more than 1,000 data points on every load and sends facility insights to shippers in real time. 

2021 highlights from across our digital freight network include:

• Shippers received to-the-minute tracking on more than 95 percent of live loads and 100 percent of drop loads. 
• Carriers have now provided more than 2.7 million facility reviews and ratings (and 40,000 emojis!), which shippers use to improve their facility operations.  
• Together, shippers and carriers avoided more than 775,000 empty miles and saved more than 2.7 million pounds of carbon emissions by shipping responsibly with Convoy. 

Now, we celebrate. Our first-ever year in review honors how each of our shippers got ship done in a year like no other. They’re receiving a hyper-personalized, data-driven look at shipment history and empty miles avoided with Convoy’s digital freight network in 2021. It’s hitting inboxes today.

The data within each year in review is just a sliver of the data shippers can access on demand in their insights dashboard, but it offers transportation teams a chance to look in the rearview mirror and celebrate all of the ship they got done with Convoy in 2021. 

What’s inside the year in review

Each year in review is unique to each shipper and unique to 2021. They can see: 

• Completed shipments – Shipments successfully delivered to their destinations.
• Miles driven – Miles driven by Convoy carriers to deliver a shipper’s freight. 
• Unique carriers – How many individual carriers moved a shipper’s loads, no matter how many times they did it. 
• Empty miles avoided – Empty miles prevented through our automated reloads program, which automatically batches full truckloads for greater efficiency.
• Carbon emissions saved – Amount of carbon in pounds saved as a result of those avoided empty miles. 
• Unique lanes – How many individual lanes a shipper ran with Convoy, no matter how many times we shipped on the lane.
• Network of trucks – Convoy shares each load to thousands of qualified carriers through our app. This number represents how many carriers viewed or bid on a shipper’s lanes. 
• Top lanes by volume – A shipper’s top three lanes by volume, which will include the pickup and delivery location and the number of shipments moved along each.
• Top-rated facility and star rating – Based on written reviews and star ratings provided by the carriers that delivered into a shipper’s facilities. More on this below.

To receive a year in review, shippers had to have moved 50 loads or more with Convoy in 2021.

A look at facility reviews and ratings

The year in review celebrates a shipper’s top-rated facility and its star rating, as nominated by the carriers on the ground. Every carrier has the opportunity to share in the Convoy app a written review and star rating of their facility experience. 

Carriers give the highest reviews to facilities that respect their time. That’s one of the insights we talk about in Convoy’s 2021 Annual Freight Insights Report, highlighting 2021 freight trends and facility insights from Convoy’s more than 2.7 million written reviews and star ratings.

Forward-thinking shippers look to facility insights as their competitive advantage. Convoy shippers can contact their account manager for a deeper look at their facility operations, or log in to their insights dashboard to see performance on dwell times, accessorial spend, and more.

Reduced carbon emissions in 2021

Each year in review shows a shipper’s direct impact on eliminating empty miles and reducing carbon emissions. Because a Convoy year in review wouldn’t be complete without highlighting our progress toward our mission to transport the world with endless capacity and zero waste.

Collectively, Convoy’s digital freight network avoided more than 775,000 empty miles and saved more than 2.7 million pounds of carbon emissions in 2021. This is progress toward a more sustainable supply chain. And yet, with each passing year, the trucking industry generates more than 72 million metric tons of carbon emissions through empty trucks. 

We have a long way to go to #NoEmptyMiles. Here’s to making every mile count in 2022.

Want your year in review for 2022? Get started today with Convoy 

New to Convoy? Get started.

Already ship with Convoy? You may have a shipper account created. You’ll just need to log in or request access. 

The post Data-driven 2021 year in review shows how Convoy shippers #getshipdone appeared first on Convoy.

Convoy Data Science (DS) teammates Carol DeZwarte and Amir Ghahari participated in the game show. Carol is also a member of the INFORMS Edelman Committee.

TL;DR:

Convoy supported four members of our DS team participating in the INFORMS Annual Meeting in October. INFORMS is one of the communities we embrace for operations research and data science. We volunteered our time to sponsor the mini competition described here, then added recruiting, interviewing, and networking. Get involved in professional communities that represent you and your teams! These are your people. They are past, present, and future colleagues, collaborators, co-authors, and friends.

The event

At the 2021 INFORMS Annual Meeting in Anaheim, Convoy sponsored and hosted the Freestyle O.R. Supreme Game Show. This event is a real-world consulting “frame that problem” competition held during one of the conference sessions. The sponsor organization presents a business problem, then participants are teamed up and work together over the next several hours to create an analytic framework and recommendations.

Each team’s competition entry is a short video describing their approach. Then a group of experienced professionals, including representative(s) from the sponsor, evaluate the proposals and choose a winner.

#NoEmptyMiles

“Sustainability in Trucking” was Convoy’s business problem for this event–a challenging space that is well-suited for a case study. We describe ourselves as a company that uses technology to make freight more efficient. Our mission is to transport the world with endless capacity and zero waste; to achieve this, our digital freight marketplace optimizes the matching of truck drivers with shippers, saving time for truck drivers, minimizing costs for shippers, while reducing needless pollution of the planet as a result of empty miles.

About 35% of miles driven by heavy trucks in the US are empty, i.e. the driver has no load. That’s 61 billion (yes, with a B) empty miles per year, so there is a lot of room for improvement. In addition to environmental sustainability, we think about other forms of sustainability, like human (e.g. flexible “green” appointment windows improve drivers’ experiences, make their days more efficient, and reduce idling) and social (our supplier diversity program gives diverse carriers access to more loads, forums, and opportunities). Learn more about how we’re advancing sustainability in freight.

Convoy also congratulates Cansu Agrali and Harsh Anand for their winning entry! We were impressed with their thoroughness, despite a lack of familiarity with our industry. Their suggestions included some work we already do across multiple forms of sustainability: optimizing routes or batches to reduce empty miles driven, optimizing networks for better asset utilization; giving teammates time and autonomy to make decisions that are good for the customers, and having health care and benefits that contribute to employees’ long-term physical and mental well-being. Cansu and Harsh also aptly noted that we should (and do) hold ourselves accountable to our goals through reporting progress against sustainability initiatives.

#BOBO

“Bring out the best in others”, or #BOBO, is one of Convoy’s corporate values. It is what we do when we make our communities stronger through participation in events like this. Convoy encourages and values community building both inside and outside our walls, and we hope you also find ways to participate in professional community events and organizations.

If you are an analytics professional and you bring out the best in others too, we’re hiring! Learn more about careers at Convoy.

The post Convoy sponsors O.R. Supreme Game Show at INFORMS 2021 Annual Meeting appeared first on Convoy.

One of our customer service standards is to notify a shipper as soon as possible of an anticipated failure to arrive on time. To enable this notification, and to give ourselves time to reschedule appointments if necessary, we built a model to predict the risk of a service failure. As is common in machine learning, there are a number of possible ways to solve this problem, but the choice of tool can have large implications on the outcome.

Regression vs. classification

From a statistics and machine learning perspective, this challenge can be addressed by either a regression model to predict when the driver will arrive, or by a classification model to predict simply whether a driver will arrive on time or late.

There are a large number of reasons a driver may be late, which makes predicting a precise arrival time challenging when something has gone wrong. How long will it take to fix the mechanical problem? If the driver is stuck trying to drop off a previous load, when will they get unstuck?

Convoy, and even the drivers themselves, sometimes lack enough information in these cases to accurately predict the time a driver will arrive. Predicting arrival times in these uncertain conditions is very challenging, but fortunately it’s not usually necessary.

For Convoy and for the shippers we work with, the risk of failure is the most crucial aspect to understand. We don’t need to know exactly when the driver is going to arrive, but we do need to know when to raise an alert and whether their appointment should be rescheduled. A classifier is more directly tied to the business problem and the outcome we want to influence. It is also the lesser included case of this problem: if we know it’s tough to predict arrival time, and we don’t actually need it, we should tackle the easier problem of late vs on time. By training a binary classifier with data labeled as “late” or “not late”, we can predict the probability of the driver being late. If this probability exceeds a threshold we set, we can raise an alert and reschedule the appointment to avoid a more significant service failure.

We can directly compare the performance of these two approaches to this business problem. In the example pictured above, both models are using the same set of features. The regressor raises an alert if it predicts the driver will arrive after the appointment time. The classifier’s threshold was chosen to match the precision of the alerts that were raised by the regressor. With the same precision, we see that the classifier detects more late shipments than the regressor does at any time before the appointment.

A classifier that uses log loss as a loss function performs better here because it more directly optimizes for the business problem and metrics of interest: precision and recall. Alternatively, the loss function for the regressor, root mean squared error (RMSE), is disconnected from the underlying business goal. Consider two examples:

1. The regressor’s prediction is two hours off, but correctly predicts that the driver will be late. The RMSE in this case is high, but the log loss is low.
2. The regressor’s prediction is five minutes off, but incorrectly predicts that the driver will be on-time. The RMSE in this case is low, but the log loss is high.

Minimizing RMSE provides us with the most reliable model for predicting arrival times, but it is not the best approach to predict whether or not the carrier will be late.

Also beneficial, with a classification model we can easily tune our alerting threshold to adjust our false positive and false negative rates. We can minimize our operating costs using the respective costs of these two types of error. This is a more challenging task with a regressor unless we have a clear understanding of its error distribution.

By choosing an approach that is directly tied to the business problem, and limiting the information we’re trying to predict to only what we need for our decision making, we gain more flexibility and a clearer picture of how to optimize this part of our business. There’s always more than one way to solve a problem.

The post Picking the right tool for the problem: Predicting service failures appeared first on Convoy.

Data and tech are unlocking new solutions in freight at a moment when supply chains are more important than ever for the economy.

At the 2021 National Association for Business Economics’ annual Tech Economics Conference on November 8th, Convoy’s Chief Technology Officer, Dorothy Li, spoke in a fireside chat moderated by Convoy’s Director of Economic Research, Aaron Terrazas, discussing the future of data in trucking, the role of data science and economists, and why reducing waste in the $800B trucking industry is so critical.

Below is a summary of the conversation, edited for brevity.

Convoy isn’t exactly a household name. For those that are unfamiliar with what we’re doing, could you tell us a little bit about Convoy?

When people traditionally think about freight and shipping, a lot of us take it for granted, but it’s really the backbone of our economy, delivering the clothes we wear and the food we eat.

Convoy’s founders had backgrounds in supply chain and were frustrated with massive inefficiencies in the $800B trucking industry, and set out to do something about it.

Realizing there was an opportunity to use data and automation to help solve the many inefficiencies afflicting the freight industry, they launched Convoy in 2015.

Convoy is an open, fully connected digital freight marketplace. What that means is, for truck drivers, they use our Convoy mobile app to find loads to haul. At the same time, we use machine learning and automation to match these drivers with shipments from retailers and manufacturers, ensuring the most optimal trucking company is selected for each specific load.

At the end of the day we’re optimizing how millions of truckloads move around the country, helping retailers and manufacturers that ship products by truck operate more efficiently, which in turn helps truck drivers earn more, and reduces carbon emissions waste from empty miles.

Coming from a background at Amazon, I’m sure you were exposed early on in your career to economists playing an active role in the day-to-day business. More and more, we’re seeing that pattern accelerate at companies. Can you tell us about your view on the role of economists and data scientists in the modern business and tech era?

My first touchpoint with an economist was right after launching Amazon Prime. The whole premise of Prime was for Amazon to measure whether it increases wallet share of customers’ spend.

But you can’t evaluate the program just by subscription volume; you have to see whether it’s actually increased the lifetime value of the subscription. Same thing with Prime Video, this is a free add-on for existing Prime members, but the content it takes to supply Prime Video is extremely expensive to make, so how do you justify it from a business perspective?

You have to see the downstream impact. When someone signs up for a subscription, how does that impact the customer’s lifetime journey? So that was my first taste of working with economists to figure out the downstream impact of these programs.

To me, economists and data scientists are not just tracking data independently, they’re working together to make sense of the various events, finding correlations, and defining the incentives.

I think it’s fair to say economists love data. Can you tell us a little bit about the data Convoy collects, and how it feeds into business decision-making?

Data and insights are becoming the currency of the modern supply chain, with transparency being at the core of transforming this industry.

At Convoy, we gather thousands of data points in every phase of the shipment lifecycle. From when it’s loaded on trailers to when it’s unloaded at a facility, we know where a trailer is at every point, and we track this in a few ways.

Our carrier mobile application uses location tracking and geofencing to proactively notify when drivers may be running late so that shippers can better optimize their labor to receive the shipment. This also helps protect our drivers by providing proof of delivery and ensuring they receive payment for their haul.

With Convoy Go, we have a fleet of smart trailers outfitted with location sensors and internet connectivity that allow us to gather routing data and optimally rebalance them for future loads.

Another thing we collect are network insights that we gather from the facilities we service. You can think about these like an Amazon Reviews or an OpenTable for restaurants; this provides a forum for truckers to provide feedback on the quality of facilities: How congested is the facility? Are there basic services like restrooms available to drivers? This helps improve the quality of service for carriers.

You’re responsible for the Economics and Science & Analytics teams at Convoy, along with our Engineering and Product groups. Based on what you’ve seen at Convoy the past couple of months, what do you think are the most interesting or exciting economics or core data science questions that the team is tackling?

Some of the most interesting problems our economists are working on are market making and designing auctions.

When you’re automating manual matching of shippers to carriers, there is a need to make explicit the incentives of all the agents and carefully build scalable matching mechanisms.

We use our economists heavily to design these mechanisms through their work in auctions, contracting, and beyond.

Another thing our economists and data scientists are doing is designing experiments.

Our economists work in tandem with our data scientists to build the best experience possible for our shippers and carriers, constantly developing and testing new features for these users with a need to innovate quickly.

To wrap things up, from your perspective, what do you think are the most important skills that new economists and data scientists need to be successful both in tech and in working with software engineers that may have a different background than many of us do?

First and foremost, being able to problem-solve with business context is critical. This means having the business IQ to understand and know deeply what the problem is to determine what your customers are actually asking for.

At Convoy we practice loving problems not solutions; and really what that means is to not fall in love with any one solution, but to look at every problem uniquely and determine quickly which assumptions are critical and which are not.

Second is recognizing data is messy. The truth is, there is no such thing as clean data. We have to deal with a lot of data systems, so it’s important to be okay with that. Companies are always improving, data tools are getting better, but be okay with data being messy and being able to adapt to that.

Third but not least, start simple and iterate, get to 80 percent quickly.

Join the Convoy Science & Analytics team

Convoy is hiring! Explore our careers page to learn more about our culture, mission, and open positions.

The post How Data is Driving a Tech Revolution in Trucking: Q&A with Convoy CTO Dorothy Li appeared first on Convoy.

Driving down the highway surrounded by heavy duty trucks is an everyday occurrence for all of us. However, the staggering fact is that 35% of the time, those trucks are completely empty. That’s a really big deal for many reasons, but it’s especially significant for the environment. Think about this: each year heavy-duty trucks run 175 billion miles moving truckload freight in the US. Of these, 61 billion are empty miles – meaning trucks traveling without loads – contributing more than 87 million metric tons of carbon emissions annually.

It’s an even bigger problem when you take into account that scientists from around the world are telling us that we have a limited window to make unprecedented headway towards the biggest environmental and societal impact we can have: staying below 1.5 degrees Celsius of global temperature rise. According to the Intergovernmental Panel on Climate Change (IPCC), the world would have to curb its carbon emissions by at least half by 2030 and then achieve carbon neutrality by 2050 to meet this target. That’s ambitious for any industry, let alone freight and trucking. 

At Convoy, we share the belief that action needs to be taken immediately. According to the Bureau of Transportation Statistics, heavy-duty full truckload freight accounts for more than 252 million metric tons of CO2 emissions per year. The $800B trucking industry literally drives the health of our economy, yet new technology-based solutions to age-old problems have been slow to materialize. Convoy set out to compare two technologies for reducing carbon emissions in freight: the adoption of electric trucks and the use of automation to batch shipments together for drivers. We wanted to understand the short and long-term opportunities of both technologies, as a way to help supply chain and logistics leaders understand how to make the most sustainable choices when planning for the movement of their freight, both now and in the future.

Our Analysis: Electric Trucks vs. Automated Reloads

Research by Guidehouse Insights found that as of 2021, electric trucks represent only about 2% of U.S. new-vehicle sales, with 30% of market share expected between 2025 and 2030. Additionally, Guidehouse’s research predicts electric trucks to be the majority of heavy duty trucks on the road sometime between 2040 and 2045. 

Unfortunately, those adoption rates may not occur fast enough to meet the near-term environmental challenges the world is facing today. Convoy’s latest research, which took the assumptions from Guidehouse and forecasted those figures over the next 13 years, shows the inflection point of when electric trucks will save the equivalent carbon emission as automated shipment batching does today, is not until the year 2034. Additionally, these findings show that supply chain leaders need to be taking action today to reduce emissions as well as preparing for the eventual adoption of electric trucks to run zero emission freight.

At Convoy, our Automated Reloads program algorithmically groups multiple full-truckload shipments for carriers, which makes it easier to find more loads to keep trucks full and earning, while eliminating wasted time searching and minimizing empty miles driven in search of work. Convoy’s Automated Reloads program has already yielded a 45% decrease in CO2 emissions from trucks running empty less often. This accounts for more than 3.5 million pounds of carbon emissions saved since we introduced bundling of loads to our network in 2019. 

If the industry as a whole were to adopt these same practices and reduce empty miles at the same rate as Convoy, CO2 emissions would be reduced by 40 million metric tons – the equivalent of taking 8.6 million passenger vehicles off the road for a year. 

Next Steps for Building a Sustainable Future 

As we come to the end of 2021 and enter into 2022, the problem is going to be the same. Everybody is still going to be talking about the impact of climate change, the supply chain is going to continue to be a disruptor and supply chain leaders will continue to be tasked with achieving sustainability goals as part of their roles and responsibilities. But, what can change in 2022 is the progress that’s made. 

In 2022, supply chain leaders should take two steps to build a strong path for net zero emissions and get empty trucks off the road. First, initiate conversations with key stakeholders in your supply chain to address the topic of empty miles – it’s one of the simplest ways to make progress when it comes to sustainability efforts. Second, start having conversations about the adoption of electric trucks and work with your carriers and partners to figure out how to introduce electric trucks into the freight network over the next 20 years. Today, 20 years may sound like a lifetime away, but it is better to prepare now. 

At the heart of it, we’re all focused on solving the same problems of waste and inefficiency in the freight industry. At Convoy, we believe that by working together to reduce the 35% of the time trucks drive empty we can make the greatest immediate impact that can benefit the environment, and that’s exciting. Join us and other forward-looking shippers and carriers as we reduce carbon waste from empty miles – the time for action is now.

The post 35% of the time that truck next to you on the highway is empty – that’s a really big deal for the environment appeared first on Convoy.

“What are the value ranges for shipper ratings? How was this dataset compiled and how often is it updated? Are there any issues with this dataset — it seems like we’re missing timestamps from last week?”

These are a few examples of the many types of queries data scientists, product managers, analysts, and other data literate individuals ask in order to answer interesting questions about the business.

Discovering data is usually a time-consuming and manual process. Data documentation in a fast-moving startup can become outdated as the business frequently changes. At Convoy, questions and answers like the above are often recorded in team-specific Slack groups or more generic analytics boards. Amundsen, an open source project from Lyft, is a data discovery tool for indexing data and serves as a centralized knowledge repository to provide high-level descriptions of internal and external datasets. However, the most interesting and relevant information about a particular column or table is often the discussions that happen between data consumers on Slack channels.

This summer, I joined the Data Platform team at Convoy to help surface common questions about the usage of our data within Slack through Amundsen. While Convoy adopted Amundsen as our platform for discovering, understanding, and trusting data, this work can be applied to any other metadata tool (i.e. Linkedin Datahub, Uber Databook, etc.).

In this post, I’ll describe the basics of Amundsen, our goals of this work, specific processes, and conclude with our next steps.

Basics of Amundsen

Amundsen is a leading open source data catalog solution. It began as an internal tool at Lyft to improve data discovery and was later added to the LF AI Foundation (Linux Foundation)[1]. The value of Amundsen comes from the ease of information access/retrieval across many types of data, which reduces the time spent by data engineers redirecting people to data sources or validating questions. For example, Lyft saw a 20% increase in productivity for data scientists and analysts through the use of Amundsen.

Amundsen collects data and metadata from many sources (i.e. Snowflake, Redshift, PostgreSQL, etc.), tags each data source, and supports the underlying mechanics for indexing and search. In many ways, Amundsen is a search engine for data.

Goals of this project

Because documentation from data sources can be outdated or confusing, our short-term goal was resurfacing prior dataset-specific questions. By allowing users to view relevant and high-quality Slack Q&As, we felt this would reduce overhead for Data Engineers (and similar roles) to answer recurrent/similar data-related questions and improve the data discovery experience.

To achieve this, we needed to ingest Slack conversations into Amundsen, which would involve:

• building an end-to-end pipeline to crawl Slack and generate the corpora,
• processing/structuring the text,
• indexing within Amundsen by modifying the frontend/metadata service and databuilder.

Secondly, we wanted to annotate conversations to related data sources (if any) and further categorize the data based on rank and quality. Since we only want to process high-quality conversations, we decided to rank all table-specific conversations, find the best reply in each conversation, and classify questions into a label set for an improved UX.

Our work and diving into specific processes

Due to the heavy API usage and Slack rate limits, we wanted our crawl feature to be an automated offline process and executed weekly as a Cron Job (time-based job scheduler)[2]. With forming the Slack corpora (left side), we used several Slack APIs to crawl our #analytics channel (where the discussions are) and then apply our ML models to further structure the data and export this information to AWS S3. In addition, we store a set of Slack-generated datasets for additional model validation and testing. Within Amundsen, we modify the databuilder and frontend/metadata service, which allows for publishing our S3 data in the tool and modifying the UI/UX of this feature.

Given our current understanding of Amundsen, goals, and a brief overview, let’s dive into the specifics!

Scraping data (specifically Slack)

Initially, our biggest concern for project feasibility was globally ranking conversations and linking them to particular tables on Amundsen. To simplify the problem space and increase conversation quality, we chose the Slack Search API as a baseline, along with an auto-filled search value. For this “search value”, we only used our #analytics channel, which holds the questions on SQL and data at Convoy. Additionally, we populated our tables on a generic pattern: {schema}.{table}.{id} (i.e. bi.shipments or bi.shipments.driver_id), which allows the linking of many responses to their {schema}.{table} metadata. Slack also supports finer-grained queries (i.e. users, emojis, …), so we can improve our autofill methods based on emojis or users who commonly answer questions related to a specific topic[3].

After generating these ranked responses for each {schema}.{table}, we use the Slack Conversations API to gather the entire thread for each response. Note that this data (with additional labeling) can be used as a train/test set for in-house models to handle overall conversation ranking and linking, along with our classification and ranking tasks. For example, I spent ~20 hours hand labeling the generated dataset to benchmark during ML architecture experimentation and also to understand the diction (i.e. formality, acronyms, structure, etc.). Since ML models rely heavily on data, a common saying is that garbage into a model means garbage outputs, which further signifies the importance of the quality of our data.

Structuring data through ML

With the crawled responses and conversations data collected from the Slack API, we tackle two questions:

1. Within a message thread, how can we find the best answer to a question?
2. How can we best segment the different types of questions to improve the UX?

For the first question, we created a ranking model to discover the most similar response to a question and integrate heuristics to further prune the possible replies. Our initial architecture was a TFIDFVectorizer to represent the text and cosine similarity to find the closest vector. Let’s unpack both of these components.

Within ML for text data (aka Natural Language Processing), the models can only compute numerical input. We need to transform the actual words/text data into numerical representations. TF-IDF (Term Frequency-Inverse Document Frequency) is one of the simplest methods to perform this task. Common words in a document lose weight while rarer and more occurring words have more weight. There are limitations with these representations such as the loss of text order/structure and semantic or contextual information. Cosine similarity is a common method to compare how similar two vectors are, and there are other methods such as Euclidean distance or dot product[4].

For the following iterations of this model, we integrate heuristics (i.e. synonyms of thanks/approvals, reactions, etc.) or either positive sentiment responses (sentiment analysis) to reduce the set of possible replies. Additionally, we are looking to improve the representation of words with contextual and semantic embeddings (i.e. pre-trained BERT), which are different vectorization methods outside of this scope.

Regarding the second question, we create a text classification model to categorize the question as either “General” or “Issue”, which can be further fine-grained for other tags. Our initial iteration uses a TFIDFVectorizer to represent the words, but instead of a similarity measure, we averaged the values of multiple classifiers (i.e. SGDClassifier, LinearSVC, …) to output a value of 0 or 1.

For further iterations, we can also use pre-trained embeddings for better representations along with other model architecture components (i.e. LSTM/CNN, Dense layer). We also incorporate user-labeling through particular Slack reactions on any crawled conversation, which is its own category.

It is important to mention that many other ML models can be applied to our problem space, such as Question Answering Systems or much broader methods in Information Retrieval. Our main bottleneck was the amount of data being generated and crawled from our sources. If you’re curious and want to further experiment with these architectures, an active research area is weak supervision (and distant supervision) which allows us to only require labels for a small subset of the data and train models in that fashion.

Integrating to Amundsen

The Amundsen integration involves modifying the databuilder and frontend/metadata services:

• Databuilder Service: We created a “job” to extract and load the AWS S3 CSV data to publish into Neo4j as nodes without relations.
• Metadata Service: We modified the table metadata query to include our Slack data in Neo4j and transfer the data to a React app.
• Frontend Service: We created a new tab component to view the structured conversations and filter by rank, column IDs, and tags.

Conclusion and next steps

After completing an MVP, we were able to deliver on our goals to have a usable and robust Slack integration through resurfacing quality dataset-specific questions. For next steps, we’ll be focusing on productionizing the prototype to gather more feedback (which is happening at the time of this writing), further improving the modeling and data quality, and looking for methods to contribute back to open source. Additional areas to explore include crawling multiple Slack channels along with other sources of conversations data (i.e. internal service tickets). We also want to build a bot that answers commonly asked questions (analyze the text and link a document or Slack thread URL).

Connecting disparate data sources with Amundsen for ease of information organization, access, and discovery would offer a lot of exciting downstream benefits. I’m excited to see what’s in store for the future. Big shoutout to my mentor Sam Lin and manager Haibo Lin for the constant feedback and help on this project! Thanks to the many coworkers who’ve helped review and provide feedback on this blog post and my project. You’ve all helped create an amazing intern experience at Convoy for me. 

— George Pu

[1] https://lfaidata.foundation/blog/2020/08/11/amundsen-joins-lf-ai-as-new-incubation-project/
[2] https://api.slack.com/changelog/2018-03-great-rate-limits
[3] https://slack.com/blog/productivity/shrinking-the-haystack-how-to-narrow-search-results-in-slack
[4] https://developers.google.com/machine-learning/clustering/similarity/measuring-similarity

The post Integrating Slack with Amundsen for ease of data discovery appeared first on Convoy.

Class 8 truck orders are one of the most widely watched data points in the trucking industry. Freight market folklore views them as a kind of canary-in-the-coalmine: When the market is hot, they foreshadow an imminent downshift.

This makes sense. After accounting for delivery lags, higher truck orders should mean that the industry is investing in new capacity — new capacity that could become overcapacity if too much of it happens or if demand unexpectedly softens. By most accounts, this is one reason for the freight industry’s notorious boom-and-bust cycle.

The reality is, truck orders reflect not only business’ plans for the future, but also the legacy of decisions made long ago. Not all truck orders are new investments; some are replacements for older, worn-out vehicles. Our analysis suggests that the trucking industry’s commercial vehicle replacement rate is likely substantially higher than conventionally assumed.

Microdata > Macrodata

Unfortunately, new trucks aren’t delivered with a sticker on the cab indicating if it’s replacing a recently retired vehicle — it’s something we have to estimate. Fortunately, this is exactly the kind of question that the data scientists and economists on Convoy’s Data Science team are trained to answer.

Most estimates of the Class 8 truck replacement rate (at least those that we have encountered in publicly available sources) rely on heuristics from industry anecdotes — for instance, a uniform seven- or eight-year replacement cycle or a long-term moving average of truck orders. From a data perspective, the reliance on high-level summary statistics is not ideal. More granular data — what statisticians call “microdata” — allows more flexible modeling and produces more reliable estimates (and is more fun to work with).

Ideally, administrative data would allow us to track individual trucks over time from their “birth” (manufacture) to their “death” (scrapping) — known as “longitudinal cohort analysis”. To our knowledge, such detailed records do not exist. A close alternative would be to observe the “age” of each active truck at a given point in time, and compare that distribution to the age distribution that would be expected given what we know about annual Class 8 truck production — a method known as “survival analysis” which is widely applied in biostatistics, ecology and demography.

We quickly realized that — by combining detailed operational and safety data that Convoy uses to assess the safety of trucking companies with publicly available vehicle information — we could create an original data set that would allow us to conduct a survival analysis, to provide a uniquely detailed answer to the longstanding question of the trucking industry’s Class 8 replacement rate.

From this insight, pulling the data together was straightforward.

• First, we took a recent snapshot of active commercial carriers registered with the Federal Motor Carrier Safety Administration (FMCSA) and state transportation departments. The vast majority of trucking companies are required to maintain federal (or sometimes state) registration, including the number of vehicles they operate. An important note is that the registrations are for trucking companies, not vehicles. 
• Next, we identified the vehicles registered to those carriers. Vehicle identifiers are available for a sample of those trucking companies that have a recent record of safety inspections. We recognize that there is the potential for sampling bias at this stage; however, as safety inspections are conducted at random, we believe the potential for bias is minimal.
• We then merged on the build year for each truck in the sample. This information is publicly available from the U.S. Department of Transportation’s National Highway Traffic Safety Administration.

From this data set, we were able to create a snapshot of the age and “birth years” of active Class 8 trucks as of March 2021. Comparing this distribution with cumulative commercial vehicle production data published by FTR, we were able to estimate a “life expectancy” range and annual replacement rate for Class 8 trucks.

Unexpected Results

Our analysis suggests that the average lifespan of a Class 8 truck is eight years, precisely in line with conventional wisdom. Importantly, however, the age distribution is not normal: The median Class 8 truck is only six years old. A handful of older trucks drive up the national average, but the “typical” truck is substantially younger.

The implied replacement rate from these results suggests that the trucking industry needs to see annualized Class 8 truck orders somewhere between 369,700 and 374,400 (midpoint of 372,200, or 31,000 per month) in order to just keep the current level of truck capacity constant. This is somewhat higher than other estimates that we’ve seen.

As of June 2021 Class 8 truck orders are running at an annual pace of 431,000 — above both replacement rate estimates, but less dramatically so in the case of ours. (Moreover, due to elevated delivery lags associated with the global semiconductor shortage, there is a longer-than-typical delay between when new truck orders are booked and when the freight market begins to see the effect of net capacity gains.)

A second implication of our analysis is that the truck production deficit is larger than other replacement rate estimates would suggest. During soft freight markets — such as the market conditions that prevailed from late 2018 through early 2020 — it is not uncommon for truck production to fall below replacement needs leading to a net decline in active vehicles. Looking purely at active truck counts may understate the true magnitude of this phenomenon since some small carriers may attempt to extend the lifespan of their vehicles during lean times with the hope of deferring large capital expenditures.

Our estimate of the industry’s truck replacement rate suggests that the capital investment deficit accumulated over the past two years totaled 39,000 trucks as of June 2021, compared to a surplus of 164,000 via more conventional estimates of the replacement rate.

Takeaways

Class 8 truck orders are an important metric to watch for everyone with a pulse on freight market developments. Understanding what portion of truck orders represents replacement — and what portion represents net growth — is critically important to accurately interpret this industry guidepost. Get the number too high and you’ll underestimate the extent of capacity growth and downside risk to the freight market; get the number too low and you set the stage for contract failure, which exacerbates market inefficiencies contributing to waste and higher prices.

In the absence of the ideal-world data we all wish we had, we must make decisions with the messy real-world data we actually have. But real-time exigencies are no excuse for napkin math. Naive estimates drive suboptimal decisions, and the predominance of suboptimal decision making creates more problems downstream. As an industry, we can and must be more rigorous.

View our economic commentary disclaimer here.

The post Survival of the Fittest: How Convoy Data Science Used Survival Analysis to Estimate the Class 8 Truck Replacement Rate appeared first on Convoy.

So, why aren’t all shipments Drop? It’s because Drop shipments require that empty trailers are made available ahead of time. That means that shippers have to predict at the beginning of each contract cycle how many shipments they are going to do so that the appropriate number of trailers can be secured. In reality, making accurate predictions every time in advance is impossible given how volatile supply chains are. During any given time period, one shipper will have excess trailers and pay penalties for underutilization while another shipper nearby has a shortage of trailers and has to push loads to live shipments.

That’s what we set out to change and it has been a long journey. In 2015, we first opened up our app to any carrier giving them instant access to live loads without needing to call brokers. In 2017, we started Convoy Go, piloting drop shipments to figure out how we could give carriers access to the large drop market. Up until then, only asset-based carriers had access to Drop. By 2019, we expanded Convoy Go nationwide and automated much of the matching and pricing.

Today, we are announcing the next evolution of Convoy Go. Our system now in real-time predicts demand and efficiently rebalances trailers across shippers to dramatically increase the ability to do Drop shipments. This means that Drop isn’t just available for a subset of contract freight, but also for backup and spot freight.

Underlying this breakthrough are four major technical innovations:

1. Opening up Drop to all carriers — The starting point was expanding drop to any carrier in the US. With Convoy’s large base of carriers that can all do drop, we can quickly bring in new power units as demand patterns shift.
2. Smart trailers & remote management — We instrumented every trailer to measure the condition, availability, and status. This lets us understand the true available capacity across our entire trailer fleet.
3. Demand prediction & automated trailer rebalancing — We built machine learning models that, on an hourly basis, predict trailer demand across facilities nationwide and then automatically determine where the trailers should be moved.
4. Automated reloads — We expanded automated reloads to drop loads to ensure that as we are moving both loads and trailers, we get maximum utilization of the trailers and reduce empty miles.

This milestone is an important step on our mission of “Endless Capacity and Zero Waste“, but we know we are still early in the journey.

The post The next major evolution of drop-and-hook shipments. Announcing Flexible Drop appeared first on Convoy.

One recent example is the conventional wisdom that the stock prices for publicly-traded asset-based truckload carriers are a leading signal for the direction of the freight market. This story sounds economically plausible — but is it true? When we tested it, we found no evidence that it holds up in the data. These results challenge a widely-held industry belief and demonstrate the importance of data and analytic rigor to a rapidly-changing trucking industry.

Why would the myth be true?

Economic logic suggests that spot market freight rates only capture the current supply and demand for trucks. They do not contain information about future market conditions. Conversely, the stock prices for asset-based carriers should reflect future expectations for the carriers’ profits. That is, stock prices contain more forward-looking information than freight rates.

Asset-based truckload carriers own their trucks and employ their drivers. Therefore, their costs do not respond to market fluctuations that drive changes to freight rates. However, their revenue does depend on freight rates. Given that profit is revenue minus cost, this means that asset-based carrier profits are positively correlated with freight rates.

Combining these two facts makes a plausible case that asset-based carrier stocks should capture future information about freight rates. However, this depends on the time horizon captured in stock prices. Changes in freight rates may be too short-term to affect investors’ decisions. Ultimately, this is an empirical question.

Data sources

To test this relationship we need two time series: one for asset-based carrier stock prices and one for freight rates. We want to avoid a situation where we have two large sets of time series and we are testing for statistically significant relationships between all possible pairs of them. For example, if we had five different asset-based carrier stock prices and ten different freight rates, and all of these were completely featureless white noise, the probability that at least one pair would seem to have a statistically significant relationship at p<0.05 is 92%. Therefore, we want to build a single series for each of the two variables in a principled way.

To develop a time series for the price of asset-based carrier stocks we use Yahoo Finance to collect weekly historical price data for the following set of stocks:

• Heartland Express
• Knight-Swift
• Marten Transport
• USA Truck
• Werner Enterprises

(Two other large U.S. asset-based carriers also have publicly-traded stock: Schneider National and US Xpress. However the time series for their prices are much shorter. Including these short-series stocks reduces the available data and makes time series analysis more difficult.) We take the first principal component to collapse variation in these five stock price series to a single time series. The first principal component explains 75% of the variance across time in the price of these stocks.

To develop a time series for freight rates we use the weekly dry-van spot rates for seven high-volume long-haul lanes:

• Atlanta-Philadelphia
• Chicago-Atlanta
• Los Angeles-Seattle
• Seattle-Los Angeles
• Philadelphia-Chicago
• Los Angeles-Dallas
• Dallas-Los Angeles

Rates on these lanes plausibly characterize the national trucking market. (Freightwaves used these seven lanes to construct its freight futures market.) Long-haul lanes are likely more responsive to national freight market conditions than local lanes. If we see a relationship anywhere it will be these lanes.

We use weekly dry-van spot rates because this is the available data set. By their nature these are less forward-looking than, for example, annual contracts. Unfortunately, no public market exists for annual contracts in the trucking industry and no data on prevailing annual rates is available. Therefore, we use spot rates as the best available data.

As above, we use the first principal component of the time series of freight rates on these seven lanes to construct a single index. The first principal component explains 90% of the observed variance on these seven routes. This shows that freight rates are highly correlated across lanes.

The following graphs show these two first principal components:

These charts might move up and down with a comparable cadence. However, visual inspection can be misleading as to actual statistical relationships. These plots do show that stock prices fluctuate on a much higher time frequency than freight rates. Some stock price fluctuations appear to have a related movement in freight-rates but the relationship is not one-to-one.

How much do stock prices predict freight rates? We know we can use historical freight rates to predict future rates. However, can we improve that prediction by adding information about historical stock prices? We can formally test this relationship with a vector autoregression (VAR). This approach jointly estimates the relationship between the stock prices and freight rates in terms of the lags of the two series.

To use vector autoregression we need to remove long-term time trends by first-differencing the series. We also normalize both series to a standard deviation of one:

As shown, volatility in both stock prices and freight rates has increased during the pandemic era. We see relatively few drastic fluctuations in stock prices. Moreover, fluctuations do not appear to be correlated with fluctuations in the freight rates.

Empirical results

As a very first pass we can run a vector autoregression where we include all lags out to four weeks. This will test whether fluctuations in stock prices and DAT rates in the last four weeks have a causal impact on current fluctuations stock prices and DAT rates, allowing for correlation between the two series. The following table summarizes the results:

As shown, the only significant relationship in this VAR model is between freight rate fluctuations last week and freight rate fluctuations this week. Freight rates have momentum; an increase last week tends to mean an increase this week too. For example, a growing backlog of shipments in periods of low capacity could explain this momentum.

However, we see no evidence that past stock prices add any information to current freight rates. Not only are none of the other coefficients statistically significant but also their magnitudes are very small. An increase in stock prices now does not foretell an increase in truck costs later.

(Nothing affects first differences in stock prices. This is exactly what we would expect. Traders’ previous decisions on publicly available information means that only true surprises can affect prices.)

This regression uses an arbitrary four-week cutoff. We can make this result less arbitrary with some model selection. Specifically we can test all sets of lags out to 52 weeks (i.e. a year) and find the model which produces the best AIC:

As shown, this model selection prefers the single-lag model which captures the momentum in freight rates. This dominates any other model out to 52 weeks of lag. Longer-term fluctuations in freight rates and in asset-based carrier stock prices do not forecast current freight rates.

Conclusion

Our results suggest that asset-based carrier stock prices do not predict future freight rates as industry folklore would suggest. However, given that stock prices reflect very long-term expectations of profitability, this is arguably not surprising. Our results do show some momentum in freight rates, which is an interesting feature of the data.

The post Do asset-based carrier stocks predict future freight rates? appeared first on Convoy.

Why TypeScript?

TypeScript is a programming language developed by Microsoft that is an extension of JavaScript which introduces typing. To understand why Convoy shifted towards TypeScript, it is important to understand why JavaScript was chosen in the first place. In 2015, the company settled on JavaScript as its programming language because of its popularity, usability, and versatility. Engineers could hop between projects with minimal setbacks since the language can be used for both frontend and backend development.

However, the inadequate code safety and the lack of documentation had caused major bugs and multiple outages. Since TypeScript was fairly new when Convoy started, JavaScript was used to reduce risk. As TypeScript matured, some engineers began experimenting with it in our codebase. Over time, the language became preferred due to its additional benefits, such as static types and improved readability. These features help engineers discover more bugs and write code with more confidence. By the end of March 2019, most engineers were eager to convert our main repository.

Before the project started, many engineers at the company considered the task monumental. Our main repository, Shipotle (a combination of “Shipping” and a burrito-making company frequented by one of our founders), had more than 75,000 lines of production code in JavaScript — many of which were legacy and business critical. To accomplish this goal, we applied our four-step migration plan:

1. Infrastructure: We added a TypeScript compiler to the build process. This allowed TypeScript files to exist in the codebase.
2. Education: We explained the benefits of the new language to engineers and we encouraged new files to be written in TypeScript.
3. Guardrails: We then blocked new JavaScript files from entering the codebase. This prevented our remaining work from expanding.
4. Migration: We explored and implemented multiple ways to convert the legacy JavaScript files to TypeScript.

After step three, with all new code being written in TypeScript, migrating files became a prerequisite for any major refactoring change. To expedite the conversion process and to split the work across engineers, an informal meeting was proposed to leadership. These “meetings” would address the technical debt in Shipotle in an incremental, collaborative, and enjoyable way. And thus, TypeScript Conversion Parties were born!

Conversion parties

The parties turned out to be hugely successful. Engineers were passionate about cleaning up the codebase and a new Slack channel was born: #typescript-conversion. Anyone could post their pull request for a review or ask questions about confusing chunks of code. With this steady effort, we began to chip away at the monolithic repository. We also started highlighting bugs, documenting best practices, and creating a giant spreadsheet to track our progress. The best part was that everyone found different reasons to attend. New employees participated because the changes were low risk and they wanted to familiarize themselves with the codebase as well as TypeScript (possibly a new language for them). Others enjoyed the opportunity to stay connected with people outside of their team. Above all else, people were motivated by the mission and the free food. We provided pizza and beverages at every party, and the parties always took place later in the workday.

Eventually, though, participation diminished and enthusiasm dwindled. People were busy with their feature work and the events were optional. Many files were also avoided for being too large, too unfamiliar, too important to mess up, or a combination of all three. Despite Shipotle being shared across many different teams and dozens of engineers, it was difficult for people to volunteer their time towards tedious tech debt cleanup. After a year, it was clear that our pace was insufficient (we were only converting a few files each week), and we would need to come up with a new plan.

Seeking other solutions

Ideas began circulating about how we would tackle the rest of the JavaScript files. One proposal was incorporating friendly competition across the engineering organization. Employees or teams that converted and merged the most lines of code could be compensated with company-wide recognition, free lunches, gift cards, or maybe even T-shirts. We ultimately abandoned this approach because many engineers at the company did not consistently use Shipotle, and thus would be adversely affected and disadvantaged by the rewards. Around the same time, people started investigating how we could automate the conversion process. We explored multiple third-party tools, and one engineer who was interested in learning more about transpilers created their own tool that could automatically infer and insert simple types. These tools, however, proved to be incomplete, and they all required manual intervention or validation. Consequently, we found ourselves persevering with the existing parties.

With the momentum of TypeScript conversion parties waning and COVID-19 forcing employees to work from home, progress on the conversion project practically screeched to a halt. JavaScript remained, though, and it was still causing problems in Shipotle. Developers worked around partial conversions, irritated as they tried to add new feature code in a safe way. Simple typing bugs went undetected such as referring to nonexistent fields or passing invalid arguments, and some linting rules (e.g. enforcing await) could not be enabled due to our mixed codebase. Yet, typing the remaining production files seemed like a monumental task.

The Shipotle Infrastructure team began to discuss how to complete the migration. Any-typing (the process of assigning ‘any’ to all variables missing a type) could quickly unlock some of the linting benefits. However, this technique would fail to identify bugs and improve developer confidence. Another approach contemplated was using automatic conversion tools. This option fell short because, as mentioned earlier, they needed supervision and nontrivial refinements. Lastly, advanced manual typing would resolve even the most complicated issues, but this method required significant effort from our developers. The following table roughly summarizes our findings from the different approaches considered:

The final push

After many discussions with engineers working in Shipotle, one engineer from the Infrastructure team analyzed how long it would take to convert a handful of files with basic typing (i.e. adding types when obvious and applying ‘any’ everywhere else). By extrapolating the time it took to convert several files to the total number of files in the codebase, it was estimated that a three-person team could manually migrate the remaining 40,000 lines of production code in approximately three weeks. The final proposal fell somewhere between any-typing and advanced typing, and included the following characteristics:

• Manually convert the remaining JavaScript files to TypeScript
• Add simple and complex types when easily identifiable
• Discover and document type mismatches or bugs
• Use the type ‘any’ for those issues and add a TODO-FIX comment

This approach would require significant development cost but it would finish the job with consistency and confidence. When approval was given, the spreadsheet for tracking JavaScript files was regenerated so the three developers could coordinate their work and progress. Excited by the momentum, other engineers began to pitch in where they could.

In the end, the final conversion push took four weeks instead of three but otherwise was a complete success. All of the production code was now in TypeScript! Looking back, we learned a few things along the way.

1. Partial conversions are inadequate. Sometimes a final push is required to reach the desirable state of completion and consistency.
2. Using tools for automatically converting types seems obvious. However, the tools will likely need oversight, adjustments, and significant investment (especially for inferring complex types).
3. Limit the scope of the project. Migrating does not mean resolving all bugs. Label some issues rather than further delay the mission.
4. Love Problems, Not Solutions (one of our company core values). To achieve success, focus on the goal rather than the process, without compromising on quality.

Finally, this endeavor could not have finished without the patience and persistence of everyone involved. Every time work stumbled and lost momentum, someone else was there to pick up the torch. Now that the migration work is complete, we can ship code at a faster speed, in greater confidence, and best of all, with fewer bugs. The seemingly insurmountable task was in fact possible and we hope this story inspires you to take on your own difficult migrations. Thank you for reading and good luck!

The post Converting a codebase from JavaScript to TypeScript appeared first on Convoy.

One of the biggest customer complaints was that the Continuous Integration (CI) process was too slow and prevented developers from iterating quickly on their features. At Convoy, we require all pull requests to pass suites of unit and integration tests as well as style checks before we allow merges into the mainline. Because there are so many tests in the monolith, it is impractical for developers to run all the different tests locally. Most developers’ workflows usually involve only running a few tests locally before relying on the CI environment to validate their changes as well as catch any regressions on the entire test suite.

When our team first set out to tackle this problem, the CI workflow took 20 minutes. All 100+ developers in Shipotle had to pay this tax for every commit they made, and it added up to limit the productivity of the entire engineering team. In addition to costing developers a lot of time, a long CI process also requires paying for more cloud compute and directly affects the company’s bottom line. Through our work, we managed to halve the CI time to 10 minutes even as the number of tests in repo increased by over 20%.

Building observability

Going into the project, our team knew that a lot of tasks would be exploratory in nature. To increase our confidence in the proposed changes, we needed to collect and analyze metrics from the CI pipeline. These metrics would also serve as an easy way to track progress over time and highlight bottlenecks that needed additional work. Luckily, the data we were looking for was readily available via webhooks from the CI pipeline. All we had to do was listen to the incoming webhooks and emit new Datadog metrics. Once the CI data was in Datadog, we were able to quickly build dashboards for easy visualization.

We ended up with two dashboards, build-test time and tests time. The build-test time dashboard gave us a top level understanding of the CI process and helped us understand trends over time. The tests time dashboard dives into individual tests and helps us determine which tests are slow and flaky.

With the two dashboards, we quickly realized we needed to focus our efforts on driving down the (Docker) build times and the integration test times.

Improving docker build times

At Convoy, we believe in running our tests inside of production-ready artifacts to minimize differences between environments. As such, building the production-ready Docker image is a prerequisite to running our test suites.

Our docker build has 4 main phases.

1. Code checkout
2. Installing the necessary node modules.
3. Compiling TypeScript to Javascript
4. Pushing the built image to our container repository

One simple trick for shaving build time off of any large and old monorepo is to leverage shallow git clones to skip downloading all of the repo’s history. We found that this simple change changed our clone times from 30 seconds to 5 seconds.

Dealing with large docker layers

We quickly found out that steps 2 and 4 are intimately related: the fewer node modules you install, the faster the push (and pull). Docker is able to push different layers in parallel, but when a single layer contains all the node modules, the slowest part of the process ends up being the time it takes to compress thousands of files before the push.

When we first looked into our node modules directory, we realized that there were a couple packages that were duplicated over and over. Yarn why helped us realize we were pinning the version of certain packages in Shipotle’s package.json, preventing hoisting from happening within many of our required internal client libraries and resulting in many nested versions instead. Just by bumping lodash 4 patch versions upwards, we were able to remove 70 duplicate lodash copies in node modules! Systematically working through our package.json list with yarn why and yarn deduplicate, we halved the size of node modules from 2 GB to < 1 GB.

We also discovered another simple trick for shrinking the size of node modules in Docker images. Yarn install caches the package tarball in a local cache directory to prevent fetching a package over and over again. This makes a lot of sense for local development but is unnecessary for a production image. By modifying our install command from yarn install to yarn install && yarn cache clean, we further trimmed down the size of our Docker image.

In addition to reducing the Docker image size, we also looked into making the Docker build more efficient. We wanted a system that could more efficiently leverage Docker’s built-in layer reuse. In particular, installing node modules over and over is extremely wasteful and slow. We rolled out a cache system that determines if the checksum of the package.json and yarn.lock files have been encountered before. If the cache exists, we pull the corresponding Docker image that will share the same node modules layer. If not, we skip the image pull, build the image from scratch, and update the cache with the new image. It does take a bit longer to pull the cached image before kicking off the build, but that is easily offset by not needing to install or push the large node modules layer.

Improving TypeScript compile times

The other main step in our Docker build is compiling our TypeScript code into Javascript. When we first started, the compile time was taking roughly 280 seconds. We tried a variety of different experiments like increasing the machine size, breaking apart the compile into smaller chunks, and upgrading TypeScript versions. Nothing worked. In the end, it came down to a single TypeScript config flag. Our configuration had the incremental flag set to true. With incremental compiles, TypeScript is able to determine which files changed since the last compile and only type check and transpile those impacted files. Developers pay an expensive one time boot up cost for faster local iteration. However, because our production artifact does not need to be recompiled again and again, keeping this flag enabled in the Docker build is useless. In fact, we actually found that keeping the flag on greatly slows down the compile time because the compiler has to do more work to output the information necessary to make incremental compiles possible. Switching the flag off immediately caused our compile times to drop down to 130 seconds.

Speeding up testing

Generally, the simplest way to speed up tests is to increase the number of containers running them. While the overall wall clock time remains the same regardless of the number of processes, there is a cost overhead for each additional container/machine we want to launch. This is because it takes time to pull, extract, and start each Docker container. While the compute cost of running more machines scales linearly, shorter test times have diminishing returns on developer productivity. Given the limited capital we can spend in this area, it is easier to view this problem as an efficiency problem instead of just a speed problem.

Tackling the slowest tests

Once we built out our test dashboard, we could easily identify the problematic slow test suites that were blocking the build. While we did discover a testing race condition that would cause some tests to get locked out and idle for 3 minutes, we found most of the slowness was a result of the gradual build up of code over time. Oftentimes there was inefficient or unnecessary setup and teardown logic that was copy and pasted between test files, and the only way to fix them was to work with the individual teams. Although the work was seemingly unglamorous (writing tests is hard enough, but reading tests is even less enjoyable), our team was able to document some common anti-patterns and implement some guardrails to help prevent future mistakes.

Improving testing container usage

Despite our best efforts to tackle the slowest tests, we were not keeping up with the influx of new testing code, especially for the integration tests. We eventually realized that we had never bothered to question the original test running strategy. The integration tests were originally set up to run as a single test process along with the required Postgres and Redis servers and this setup had never been revisited. A quick ssh to one of the test containers and we saw that the container was being underutilized!

After that discovery, we experimented with running multiple isolated test processes via backgrounding, passing each test process its own unique Postgres database and Redis server to maintain isolation. As we tweaked the number of background test processes to run inside each test container, we closely monitored our dashboards to understand if we were causing the CPUs to thrash or if we could push the machine harder. We found our sweet spot to be 5 background test processes (and their corresponding databases) running on a 3 vCPU machine. Before backgrounding, our integration tests were consistently taking 9–10 minutes. With our current setup, the tests take about half as long and sometimes even finish in less than 4 minutes.

Working in, supporting, and optimizing a large monolithic code base can be challenging in the best of times and it can begin to feel like the legacy systems are actually slowing down progress. Although it took time for our team to get familiarized with each corner of the monolith and begin to establish a broader domain expertise, by digging in so deeply, we were able to uncover simple, high impact fixes that greatly improved the CI pipeline.

Through this work, we discovered three key takeaways:

• Observability and transparency are critical when pushing forward a difficult project
• Sometimes it’s the smallest changes that make the biggest impact but only by knowing the code base intimately could we root them out
• Perseverance and a little out of the box thinking can be key to uncovering new solutions

Hopefully hearing more about our process has been helpful and you can apply some of these tricks to your CI pipeline as well!

The post Improving continuous integration for our monolithic codebase appeared first on Convoy.

Fuel had two objectives:

1) Create a visual and interactive consistency system where our broad spectrum of internal and external products could be unified

2) Redefine the way engineers develop new UI features for our products

We wanted to build a tool where developers could be more agile and build amazing features faster without focusing too much on the little UI details. In this post, I’ll talk from an engineering perspective about the challenges, steps, and learnings the Web Platform Engineering team took to build Fuel.

We know what we want, let’s get to work. Not yet!

From the start of the project, we knew what we wanted to achieve with Fuel and had a general idea of what success looked like. It seemed like a very straightforward component CSS project. It’s very tempting to jump right into development when you are given an exciting project like Fuel (and even more so if you are passionate about UI engineering like me), but I’ve seen projects go wrong or be limited in impact because they lacked planning.

What about the developer experience? What were the developers’ current struggles? What technologies were the right ones to use? Should we build it from scratch, revamp our pre-existing library, or use a 3rd party library? What components were we building and in what order?

In short, we had many questions we needed to answer before jumping into any sort of development for Fuel.

Bring in the tech

We wanted to validate our assumptions on our initial technology choices. One fundamental characteristic we looked for was highly adopted technologies from around the web developer community. Some benefits of these technologies included having reliable documentation, feature improvements, and community collaboration with solid support.

At Convoy, Typescript and React were already heavily used, and we knew developers had a good experience working with these technologies. Having typed components would accelerate their development and help reduce runtime errors in production. We knew that going with a Typed-React component approach would not only be beneficial for everyone but would also reduce the ramp-up time to learn these technologies while adopting Fuel.

Additionally, we needed to figure out the right CSS library for our project. We knew from previous feedback that developers needed a better way to restyle components. A few engineers suggested Emotion styled-components. Thanks to its technology under the hood (like Stylis), its styled-component approach to reusability, and how easy it was to manage styles, we decided Emotion styled-components were the right library to use in Fuel.

With that, we had selected the technologies we wanted to work with. We had the perfect combo, Typed React Styled components!

Should we build from scratch?

We decided to select our technology bundle first because we wanted to make sure we focused on our main objective: to improve the developer’s experience. Once we found the technologies that made the most sense for our teams, we had to analyze if we should build it from scratch, revamp our pre-existing component library, or use a 3rd party library and rebrand it.

To figure out the best path, we did an assessment of our effort costs. A summary shown in the table below shows how we split the work that had to be done into two categories:

• Unique Effort: The amount of effort each of the options would take to complete
• Common Effort: The amount of effort which was shared across all the options. This means that no matter which option we selected, this work had to be completed

This table helped us visualize what the different scenarios looked like. Building from scratch had the least amount of complexity and technical burden. The other two options had components already built for us, but not with the tech stack we wanted, and would require additional work. On the other side, building from scratch allowed us to have our tech stack built into every layer of Fuel and avoid any hacks and tweaks required for pre-existing components. For us, the best investment was to start Fuel from the ground up.

Component roadmap

Now that we knew we were building components from scratch and that we were going to need to develop initial components, we started creating a component roadmap strategy that would allow us to build components in the right order.

We decided to build initial components that had no other component dependencies. For example: Typography, Form Elements, Icons, and Buttons, including a grid system for responsive design. This was the base foundation from which all other components were going to be built.

We prioritized more complex components based on the upcoming features roadmaps for different products.

Can we code now? Yes!

We now had everything in place and a solid plan. It was time to start our alpha version development. The design team had previously designed the initial components in the Fuel design language, so we had the proper requirements on how components should look and behave.

Throughout the building phase, both Design and Web Platform teams met constantly and consulted product teams about scenarios and perspectives on how components were going to be used.

Getting teams involved

We wanted our platform to speak for itself and have people adopt it naturally versus having any sort of forced top-down mandate to migrate from management. For Fuel, adoption happened a little after we hit our three-month mark of starting development. We wanted to hold off sharing Fuel until we had enough basic components where teams could start building important elements for their web apps and experience a little taste of the Fuel experience; think of it as the MVP.

We decided to have the MVP ready for our yearly hackathon week, “Moonshot”. As it was coming up soon, we thought it would be a great way to test our first version. We worked hard during the upcoming weeks to get Fuel ready to be tested during our company hackathon. This opportunity would also allow us to validate our goal of improving both developer velocity and experience. We knew getting feedback early and often from our initial adopters would put us in a great position to succeed later.

A handful of teams decided to start their projects with Fuel during that week. These teams provided very useful feedback on the components, design, and our documentation examples. Early adoption allowed us to make sure the direction and development expectations for Fuel were aligned with what we wanted to deliver to our developers.

There were two very important key factors that helped boost Fuel’s adoption: People and Communication.

People knew about Fuel and its goals thanks to the conversations we had with different teams during the building phase. Designers took the task of creating new features using the Fuel design language which helped push engineers to adopt Fuel into their products.

Communication and transparency kept everyone aligned and updated. By utilizing a common Fuel Slack channel, designers and engineers could post questions, ideas, and feedback. Everyone could chime in and give their opinions. By having transparency on what was going on, the patterns we were following, and when bug fixes were being delivered, the community understood where we were taking Fuel.

Our community quickly evolved at Convoy, engineers loved the new design language, the smoothness of the UI and how easy it was to work with it. Developers from even more teams started to contribute to Fuel making it bigger and better.

After 10 months of development, Fuel is now finalizing its beta phase. It has been adopted by multiple products while continuing to grow the number of useful components it has. Having Fuel as a common foundation has made it even easier to start sharing components across products. And thanks to the engagement of all designers and engineers who have contributed and provided great feedback, we are just weeks away from a Fuel v1.0 release.

Learnings from our journey

Here are a few other learnings to consider when designing and developing a component library:

Never forget your customer — Different people with different needs will consume your product. Don’t make assumptions. Consider talking to your customers on how they are going to use your product. Keep things flexible enough for others to make adjustments or get creative with your components.

Always have the KISS principle in mind — Don’t over complicate things. Components are easier to use and test when an expected I/O exists. Keep It Stupid Simple.

Keep it consistent — Props and behavior should be consistent throughout the library. Developers will tend to use one component similar to other ones, so keeping a consistent behavior for things like animations, transitions, and hotkeys with a similar API structure and using common naming conventions is very important.

— For example, if you use the Esc key to dismiss, make sure to allow this on all other components that have a similar behavior.

Scale components properly — When developing your initial components, make sure to leave any logic based on props to a more complex component. This means if you need to handle different behaviors based on props values, you might try rethinking your component. It could be tempting to make components behave differently in different scenarios, but leave that layer to more complex components. Make your foundation as logicless as possible.

Hackathons are a great place to test —

1. It has the right audience
2. It’s a small group
3. Easy to get feedback and there isn’t much risk for developers
4. Able to trace down issues and take actions quicker

There are some key benefits of having a design system being adopted throughout a company. It enables a more successful cross-collaboration strategy between team members and products. It also allows engineers to easily switch between projects and cross-train team members, reducing the ramp-up burden.

Overall, Fuel has delivered great improvements not only for customers but for our developers as well. Having pre-built and extendable components that just work out of the box has allowed developers to easily connect logic to the UI, build features faster, and ship impactful improvements to our customers quicker. From an engineering standpoint, this project was a great opportunity to take two steps back and rethink our components and patterns.

While we made choices based on what was best for our teams and our developer experience, there are other options that might be better for other teams and companies. My advice is to be thorough, make your assessments properly, provide value, and work to get your teams fully on board. I hope this story provides you with a few insights and helps your team’s decisions.

The post The journey, building Convoy’s component library appeared first on Convoy.

This proved to be the case when Anheuser-Busch awarded Convoy its Non-Asset Carrier of the Year Award, a recognition typically bestowed upon traditional freight brokers. Today, Convoy is one of Anheuser-Busch’s largest over-the-road carriers.

While Convoy’s technology platform and data provide a strong foundation that equips us to make better decisions, it’s the level of service, outstanding performance, and the tailored programs that has earned Convoy a deep level of trust that has taken our partnership far beyond a tech platform.”

Anheuser-Busch InBev’s VP of Sustainability and Logistics Procurement

Why would more technology lead to higher levels of customer service? Let’s look back at history for an example of how technology automation helps us spend less time on transactions and more time on higher-level consulting.

How a DFN is like an ATM

Consider the impact of the automated teller machine (ATM) on the banking industry. When ATMs were introduced in the 1970s, people worried that this would lead to massive unemployment for bank employees. Instead of going to a human teller, a customer would use a debit card at an ATM to withdraw and deposit cash. 

Surprisingly, the opposite happened. As ATM use exploded, the full-time equivalent workers at bank branches grew steadily. Economist James Bessen covers this in his book Learning by Doing: The Real Connection between Innovation, Wages, and Wealth.

ATMs didn’t eliminate the need for tellers. They changed the way tellers work. Today, tellers spend less time on simple money-counting transactions well-suited for machines, and more time on complex customer support issues best-suited for people. Meanwhile, bank branches expanded the role of financial advisors, loan officers, and business banking specialists. In this case, automated technology didn’t remove the human connection, it deepened it. 

Our digital freight network follows the same model. We use machine learning technology to automate the freight transportation process, including tendering, matching, and hauling. A traditional broker or asset-based carrier may take hours to coordinate a truck with a load. Our platform can achieve this in minutes instead of hours without a single phone call or email. 

We spend less time on the operational tasks suited for technology, and more time on higher-value strategic questions better suited for people. Time that was once spent emailing, calling, collecting bids, and negotiating rates is now focused on analyzing your lane network, evaluating facility performance, and planning capacity for your shifting needs. This helps lower transportation costs for shippers, reducing incidentals and wait times, while providing reliable and flexible carrier capacity. 

Just as ATMs changed the role of a banking branch, a DFN is changing the role of a truckload freight partner from being transaction-oriented to people-oriented.

Supply chain consulting with account management and data science

In addition to automating the freight matching process, our digital freight network collects thousands of data points on each shipment we complete. We’ve invested in building a world-class account management team, with account managers who have deep experience and industry-specific knowledge across verticals. We’ve also assembled a data science team with PhD-level experts who are dedicated to solving supply chain challenges through network insights.

We provide every shipper in our network with automatically generated reports. These highlight trends and anomalies that may warrant further investigation. We also provide an online reporting dashboard that gives shippers flexibility to customize and visualize your data based on specific timeframes, geographies, shipment types, and more. 

Beyond automated reports and online tools, our account management and data science teams provide custom insights and solve supply chain challenges through collaboration. Together, we’ve worked with shippers in every industry to improve their supply chain efficiency, reduce costs, and improve their customer service.

Supply chain visibility on a massive scale

Convoy’s digital freight network provides the most comprehensive data set in the industry.  We leverage our comprehensive data on nationwide lanes, locations, and drivers, and combine this with institutional knowledge and familiarity with the unique needs of our shippers to provide meaningful, actionable insights. 

If you’re interested in exploring how Convoy’s network insights and supply chain consulting could benefit your business, we’d love to make an introduction. In as few as 100 loads, we can examine patterns in your shipments and identify potential areas to improve your transportation efficiency.

More resources on how digital freight networks can support supply chain visibility and consulting:

• Download our white paper: Supply Chain Visibility and the Digital Freight Network.
  
  
• Whether you’re selecting a freight partner or starting a freight RFP process, consult our guide to developing a freight transportation strategy.
  
  
• Discover three ways supply chain visibility can improve facility performance.

The post Supply chain consulting from a digital freight network appeared first on Convoy.

The combination of micro and macro-level insights makes supply chain visibility a powerful tool for shippers. If you’re moving thousands of truckloads a year through a facility, even a minor recurring issue can lead to significantly higher incidentals, lower carrier satisfaction, and higher total cost to ship.

To provide shippers with a deeper level of supply chain visibility, Convoy’s digital freight network automatically collects more than 1,000 data points on every shipment. Convoy also enables carriers to provide detailed feedback on their experience at facilities. We then generate automated reports for shippers in our network, and our team of data scientists analyze information to equip shippers with insights that can improve the efficiency of their freight logistics. 

In this blog, we highlight three examples of how supply chain visibility helped solve facility issues faced by Convoy customers. 

Note: The following are real problems faced by real customers, however we’ve anonymized all data to protect our customer’s privacy.

Question 1: “How can I reduce detention costs at a problematic facility?”

The Problem: You’re struggling with dock bottlenecks and backups. Anecdotally, you can’t point to any specific problems or patterns.

The Process: The data uncovers that detention begins to increase at 3AM, builds to a peak between 6 and 7AM, and returns to normal by 11AM. Once time of day is identified, we look at shipment volume and detention cost to uncover that you’re effectively paying an extra $40 per load during the peak hours of 6 to 7AM.

The Insight: If you shift your morning picks and drops across the afternoon and evening, you could save up to $150,000 per year.

Question 2: “What caused a jump in my month-over-month incidentals at my top facility?”

The Problem: You noticed an unexpected 23% MoM increase in incidentals at your highest trafficked facility, according to Convoy’s monthly business report.

The Process: Convoy’s investigation discovers that wait time at the same facility nearly doubled in the same timeframe, and volume increased by almost 20%.

The Insight: The volume is taxing the facility’s ability to get trucks in and out, increasing detention—the number one incidental contributor. As a next step, we look to do load balancing and rerouting to other facilities where it makes sense.

Question 3: “I have a handful of facilities in the same region—why is one responsible for a disproportionate amount of incidentals?”

The Problem: One of your facilities experienced more incidentals in a single quarter than all your other facilities combined according to Convoy’s monthly business report.

The Process: Convoy’s data science team discovers that the facility has limited shipping windows compared to other similar-sized facilities in the network. These narrow shipping windows are the result of the facility being staffed by only five lumpers—roughly half as many as are employed at your comparable facilities.

The Insight: When schedules go awry (as schedules tend to do), the facility employees have no stretch to put out fires while also keeping the facility running at the same time. Now armed with this information, you’re empowered to make the case for increasing people resources at the understaffed facility.

Achieving supply chain visibility with a freight partner

These are just a few examples of how Convoy has helped shippers solve improve their freight logistics. While Convoy’s data and technology may seem like the star of this story, the unsung hero is our close collaboration with our clients. 

We collect more than 1,000 data points per truckload shipment. Turning this data into insight requires a deep understanding of our partner’s business and the unique characteristics of their freight. In these examples, our partners had a dedicated account manager with intimate knowledge of the shipper’s business paired with Convoy’s team of PhD-level data scientists. This combination of the data, technology, and collaboration represents a new form of freight partnership that can bring visibility to your supply chain.

For a more in-depth picture of how supply chain visibility can improve efficiency in your operations, download our white paper: Supply Chain Visibility and the Digital Freight Network.

The post 3 ways supply chain visibility can improve facility performance appeared first on Convoy.

For many shippers, business insights like these aren’t readily accessible, creating a blindspot in their supply chain operations. For companies who ship with Convoy, this critical information is always available through the use of our network insights products and services.

Turning Data into Business Insights

Network insights track the lifecycle of your freight – the journey a load takes from initial tender through pickup and all the way to delivery and carrier payout. Along the way, our digital freight network collects more than 1,000 data points about your facilities, wait times, incidentals, shipment performance, and more. We then use machine learning to analyze the data, and with the help of our data scientists and supply chain strategists, we surface business insights to our customers’ logistics teams through reports, online tools, and custom engagements, offering a level of supply chain visibility never before possible.

This type of information can be particularly valuable when navigating supply chain disruptions and volatile market conditions like those created by the COVID-19 crisis. In a matter of weeks, our industry saw tightening and softening market conditions that normally play out over the course of many months. And during this time, Convoy had the privilege of partnering with our customers in every industry, helping them understand how their supply chains were responding to the demand shocks, and recommending changes to make their systems more resilient.

Responding to a Demand Surge

As the COVID situation escalated in early March, one of our shipping partners experienced a massive flood of customer orders, causing weekly tender volumes to double. Fortunately, Convoy’s digital freight network was built for such situations. Our tens of thousands of carriers were able to swarm to the areas of greatest need, absorbing much of this shipper’s surge and accepting 30% more tenders than in pre-surge periods without compromising on on-time delivery (OTD).  

More importantly, we were able to use data collected from thousands of shipments to show how the shipper’s supply chain was overwhelmed by the sudden surge in volume, the costly downstream operational impact, and how operational bottlenecks could be avoided in the future.

When the deluge of customer orders came in, our shipper’s tendering system automatically created shipment requests to fulfill each order. Unfortunately, the number of shipments created far exceeded their facilities’ loading capacity for the period, causing the percentage of shipments with blocked appointment requests to double. This created a domino effect. Loading facilities got backed up as they scrambled to work carriers in, extending wait times by an average of 90 minutes. Pushed pickup appointments and long loading delays made  “requested delivery date” (RDD) failures twice as likely, and cascading operational issues caused a 40% increase in incidental cost per shipment.

Once we’d identified the root cause of the problem, we were able to recommend changes that would improve the durability of the shipper’s tendering system. Specifically, we recommended building failsafes into the tendering system that would prevent the automatic creation of shipment tenders when order volume exceeds a given threshold. This would give the shipper greater control over their network during volatile periods and allow them to be more strategic in considering the tradeoffs between volume and operational efficiency. 

A New Kind of Freight Partnership

Each of our customers has a supply chain and set of operational processes that are unique to them. And the logistics professionals we’re lucky enough to work with each day know the intricacies of their transportation systems in a way we can never hope to. 

However, through the use of thousands of data points collected on every shipment, machine learning, and dedicated Convoy team members, we’re able to partner with shippers to uncover hidden inefficiencies in their supply chains and help them identify ways to improve their operations, lower their shipping costs, and improve the resilience of their systems. It’s one of the things that differentiates a digital freight network from traditional brokers and asset-based carriers, and it’s one of the most rewarding parts of what we do at Convoy.

With market volatility increasingly looking like the new normal, a digital freight network can help you optimize your operations and respond more quickly to demand shocks. As Gartner analyst Bart De Muynck recently wrote, “Digital freight networks can help companies that are looking for real-time available capacity or looking to reduce transportation costs during the current crisis, as well as during future challenging times.”

If you’re looking for new ways to build resilience into your supply chain, we’d love to help. Just fill out the form on this page and a member of our team will reach out to learn more about your specific needs. Or if you’re interested in learning more about the types of supply chain insights that Convoy could provide, check out our latest white paper, Supply Chain Visibility and the Digital Freight Network.

The post Improving Supply Chain Resilience with Shipper Insights appeared first on Convoy.

It comes as no surprise that April was a brutal month for the U.S. economy — industrial production, factory capacity utilization, and retail sales all posted record contractions. This comes on top of March, which itself saw the biggest monthly decline in factory output since the end of World War II. 

We’ve known for some time that April’s data would be rough. Anyone who went to a grocery store last month knew that intuitively. Real-time truckload demand over the course of April showed a deterioration in commerce following the early-March surge, then a steep drop as the economy went on pause, which stabilized at a low level only toward the last week of the month.

Some categories moved sharply; others hardly fell at all (or, in a small number of industries, increased).

On the consumer side, retail sales at online stores jumped 22 percent and grocery spending was up 13% from a year earlier while clothing spending fell by nearly 90% and spending at electronics stores was down by close to 50%. All-in-all, U.S. consumers spent about $130 billion less in April than the pre-COVID-19 trend suggests they would have otherwise, and that was on top of about $50 billion in lost retail spending in March.

On the factory side, output of durable consumer goods — mostly big-ticket items like computers, cars, and washing machines — fell 36% in April from March (down 45% from a year earlier), worse than the worst months of the Great Recession and touching its lowest level since 1992. Motor vehicle and parts output fell by 72% from March (down close to 80% from April 2019); factories in the automotive sector ran at around 15% capacity in April. But output of nondurable consumer goods — mostly essential household items like processed food and cleaning supplies — fell only 5.5% (down 7% from a year earlier).

May is already revealing itself to be the opposite of April, though tentatively so. In Convoy’s network, truckload demand increased modestly the first week and more sharply during the second week of May. It’s still only about 80% of where demand was in February, but it has very clearly turned a corner.  Truckload demand from consumer staples shippers in Convoy’s network is now above where it was at the end of February and demand from food and beverage producers is only slightly below. Industrial manufacturing is just beginning to restart, and is still 15%-20% below late-February levels. 

Convoy’s Freight-Weighted Industrial Production index — which weights industrial production and retail sales by their truckload content to provide a snapshot of aggregate truckload demand — fell to its lowest level since January 1994 and remained in recession territory in April, as it has been for over a year and a half now. That is likely to continue for the foreseeable future.  

View our economic commentary disclaimer here.

The post April Industrial Production and Retail Sales: A Window Onto the “Essential” appeared first on Convoy.

A Small Decision

Let’s deep dive into a real-life example found when we improved the Convoy experimentation service, which helps product teams conduct controlled experiments or A/B tests. On the surface, making control vs. treatment assignments should be a trivial task. But, the quality of the assignment process had a fundamental impact on the outcome of the experiments.

Previously, our assignment algorithm worked like this:

1. Data scientists specified any positive integer as the weight for each variant of the experiment. For example, let’s set

the weight of control: W(c) = 4the weight of the treatment: W(t) = 1

2. We summed up weights of all variants to get total weight,

W = W(c) + W(t) = 5

3. We generated an integer H = Hash(exp, e) for entity e in this experiment exp, say

H = Hash(exp, e) = 987654674

4. We used the modulo function

Mod(H, W) = 4

5. Finally, we randomly assigned integers in [0, W-1] to control or treatment according to the weight ratio.

6. For this example, let’s assume mod result [0, 1, 2, 3] would be assigned to control and [4] would be assigned to treatment. Hence, we would assign entity e to treatment for this experiment.

This algorithm worked fine until the product teams took a gradual rollout approach to mitigate the risk of bad experiments. In this iterative approach, the product teams wanted to put a small percentage of the population on treatment and leave the rest in control in the initial version. They wanted to shut down the experiment early if the treatment showed disappointing results. If the treatment looked reasonably good, they would shift more population to treatment. We identified each such shift as a new version of the same experiment.

When this experiment went from v0 to v1, the mod function Mod(H, W) gave a different result for the same entity e. There was no guarantee that entity e would be assigned to treatment in v1. But an entity previously assigned to treatment should not be reverted to control while we roll out treatment to more entities. More importantly, the experiment results would get polluted due to these flips.

To avoid flipping assignments, the experiment upgrade process had to decide which assignment decisions to carry forward to the next version, and which to overwrite due to the weight change. This memorization based process also could not survive over two generations. Things got even more complicated if an experiment had more than two variants.

This cross-version assignment process was nondeterministic and nearly impossible to validate. Bugs were hard to investigate and fix. The complexity also rendered the system more fragile and subject to operational issues in the production environment.

After investigating a few assignment quality issues, we redesigned our assignment solution for full transparency with two dead simple steps that are repeatable and verifiable.

• The new solution introduces a concept of buckets and requires consistent total weights.
• In step one, each entity will be randomly hashed to one of the 100 buckets. The bucket an entity gets assigned to will likely be different for different experiments but will be consistent across different versions of the same experiment. Step one is random but verifiable.
• In step two, our algorithm allocates these 100 buckets to different variants. When users roll out an experiment from one version to the next, our algorithm only needs to reassign buckets. This picture below illustrates how bucket allocation changes when the experiment went from 90/10 control vs. treatment to 50/50.

Each of the two steps above is transparent and auditable. While simplicity resulted in a more robust system, we also reduced storage and serving cost because we no longer need to carry individual assignments forward to the next version.

But wait, how would this work for experiments with more than 2 variants? For example,

How were we going to change the bucket allocation when upgrading from v0 to v1? Would this work?

The population in the T2 buckets of v1 would include some entities from T1 of v0. This was not acceptable because these entities would have been exposed to both treatments and that could invalidate the experiment results.

To solve this problem, we came up with this new approach which allows us to scale up and down comfortably until we are ready to declare a winner among the treatments. Each treatment will have a dedicated range of buckets to expand and retract. This guaranteed that an individual entity won’t be exposed to multiple treatments before a winner is declared.

To accommodate more complicated scenarios, bucket allocation could also be edited manually. Notice that manual allocation of the buckets would not affect the randomness of the assignments because step one already took care of the randomization at the experiment level.

Love harder problems?

If you enjoyed this deep dive, consider joining Convoy. We are actively advancing technology to address the hard challenges of making assignments. For example, here are a few deeper assignment problems that are relevant to a company like Convoy with a unique marketplace.

• How could we improve our stratified sampling over a skewed population? Experiments across users with extremely imbalanced characteristics can have impacts that invalidate results.
• How do we reduce market interference when the population in control and treatment interacts with each other in a shared marketplace? The unfair competitive advantage of a treatment group over control might invalidate any metric improvements.
• How should we make assignments that help differentiate algorithmic improvements versus the human effect of operation teams whose actions might affect metrics? Existing operation processes might require human interaction on groups in different treatments and these processes are difficult to change.

Convoy is on a mission to reduce billions of miles driven empty in the freight industry. Innovation through experimentation is how we roll. Improper assignments waste precious engineering resources, slow down the experimentation flywheel, or even lead us down the wrong path. How we do assignments is a small decision with a BIG impact!

The post Improving the experimentation process: Small decision, big impact appeared first on Convoy.