Skip to main content
Strategy

From Dial-Up to Data Lakes: The Nature of Digital Business

How easier access to infrastructure changes the costs, dependencies, and responsibilities of digital business.

Cloud services, APIs, and data lakes make new capabilities accessible. Their business value depends on managing consumption, dependencies, data meaning, and recovery.

From Dial-Up to Data Lakes: The Nature of Digital Business

A retailer introduces online ordering using a hosted storefront, a payment provider, and a managed database. The team can assemble the service without building every capability itself. Later, an order is charged but not confirmed, cloud costs rise faster than sales, and two reports disagree about customer numbers.

The services still provide useful capabilities. The retailer now needs to understand how they behave together and who owns the gaps between them. Buying access to a capability changes the work required to operate it within a business process.

The movement from dial-up connections to cloud infrastructure and data lakes helps explain this relationship. Improvements in connectivity and infrastructure make more products practical. They also change which costs and constraints teams encounter, and how easily those constraints can be seen.

Connectivity Changes What Applications Can Assume

Dial-up connections made transfer time and intermittent access visible to users. Software had to account for limited bandwidth and the possibility that a connection would not remain available. Faster, more persistent connectivity made richer online interactions practical, although users and locations still differ in the access available to them.

For a retailer, this can support continuously updated catalogues, remote payment services, and order information available across several locations. Each feature introduces interactions whose timing and failure behaviour matter to the customer experience.

An order request might call inventory, payment, and delivery services before returning a confirmation. Even when each interaction is reasonably quick, the complete operation can be slow. If one service is unavailable, the application needs a defined response.

Connectivity therefore belongs in the design of the business process. The team needs to know which actions require an immediate answer and which can remain pending. Assuming that every remote request will complete makes the normal path easy to demonstrate while leaving the interruption cases unresolved.

Cloud Changes How Capacity Is Obtained

Cloud infrastructure allows a team to obtain computing and storage services without buying the underlying equipment for each application. This can reduce the physical commitment needed to test an idea and make capacity changes easier for workloads designed to support them.

It does not establish unlimited capacity. AWS service quotas, for example, apply to services and are generally specific to a region. Some can be increased by request; others cannot. Applications also have their own limits involving connections, concurrency, storage performance, or architecture.

The retailer should therefore test the complete ordering path against expected demand. Increasing application instances will not resolve a bottleneck in a payment integration or a database that cannot accept more connections.

Consumption also needs ownership. Test environments, retained logs, and unused resources can continue generating charges. Teams need a way to relate the bill to services and activity, then decide whether the expense is justified. The infrastructure strategy should account for these operating responsibilities alongside the flexibility gained.

APIs Put Capabilities Behind Service Boundaries

A payment API gives the retailer access to specialist infrastructure without requiring it to build a payment network. Similar interfaces can provide messaging, identity, maps, and other capabilities. The team can then devote more effort to the parts of its service that distinguish it.

The integration still needs to account for dependency failures. A timeout may leave the retailer uncertain whether a payment was accepted. Retrying without understanding the provider’s behaviour can create another problem, while immediately declaring failure may give the customer incorrect information.

The business needs a way to identify the operation, establish its status, and resolve uncertainty. Those requirements should influence the provider contract, application design, monitoring, and support procedure.

This is a practical boundary for ownership. The provider operates its service; the retailer remains responsible for how an uncertain result affects an order and what the customer is told. An interface simplifies access without specifying every business consequence of using it.

Data Lakes Make Retention More Flexible

As businesses collect more events, transactions, and documents, they may want to retain information before every analytical use is known. A data lake can hold structured and unstructured data in forms that support later processing and analysis. It need not require every source to conform to one analytical model before storage.

For the retailer, this might preserve order events, support messages, and catalogue history for investigations that become useful later. Retaining source information can allow a team to revisit earlier interpretations rather than relying only on a report prepared at the time.

The ability to keep the data leaves questions about permitted use, retention, and access. It also creates a requirement to record where information came from and how it has changed. A large collection is useful when people can identify the data relevant to their question and understand its limitations.

A data lake can include validation and curated datasets. The choice is about where particular checks and transformations belong, rather than whether structure and quality matter at all.

Reliable Meaning Requires Decisions

Suppose one report counts every registered account as a customer while another counts only accounts with a completed purchase. Both calculations may execute correctly and still produce different totals. Moving the records to shared storage does not settle which definition the business needs.

Other differences may be technical: one source uses seconds for a timestamp, another milliseconds, or two systems assign different identifiers to the same account. These distinctions need to be understood before the records can be combined reliably.

An enterprise data strategy should establish who defines important concepts, which sources control particular facts, and how consumers learn about changes. Quality requirements should follow the intended use. Data suitable for a monthly trend report may be too old for an inventory decision.

For the retailer, the useful outcome is a report whose audience understands what is counted, which period it covers, and who investigates a discrepancy. Storage and analytical tools support that outcome when the definitions and responsibilities are maintained alongside them.

Distributed Work Needs Recovery Behaviour

The charged but unconfirmed order exposes a coordination problem. Payment, inventory, and order records may belong to separate services, so the business operation may not complete as one local database transaction.

The team needs to define what happens after each partial result. An unavailable item might require a refund, a delayed confirmation, or staff intervention. Microsoft’s saga pattern guidance describes coordinating local transactions and compensating for completed steps when a later step fails. The appropriate compensation still depends on the business rules and may require further recovery if it fails.

Investigation also needs context across service boundaries. Distributed traces can connect the operations involved in a request, helping a team locate where time was spent or a failure occurred. They complement application records that establish what happened to the order and payment.

The retailer needs both technical evidence and an operational route for resolving the customer’s case. A dashboard showing healthy servers will not establish whether an individual order completed correctly.

Choosing Which Complexity to Manage

Abstractions allow teams to use a capability without understanding every implementation detail. Joel Spolsky’s discussion of leaky abstractions explains why underlying behaviour can still become relevant. A hidden limit or failure may require knowledge beyond the interface a team normally uses.

For a digital business, the decision is whether the capability gained justifies the cost, dependency, and work around that boundary. Operating a payment network internally would rarely be a sensible response to integration complexity. Defining timeouts, reconciliation, and support ownership may be.

The retailer can evaluate its architecture by following a real order through completion and recovery, then examining the resources and data needed to support it. That connects infrastructure choices to an outcome the business can verify. As services and demand change, the same approach helps the team decide which responsibilities it can delegate and which it still needs to understand directly.