Skip to main content

5. System Design Principles (Middle+)

Middle​

What is System Design and why is it important?​

Answer: System design is the process of defining the architecture, components, modules, interfaces, and data for a system to satisfy specified requirements. It is crucial because a well-designed system can scale efficiently, handle failures gracefully, and be maintained cost-effectively. Poor design can lead to massive financial losses and system-wide outages.

What is the difference between Functional and Non-Functional requirements?​

Answer:

  • Functional Requirements: Define what the system should do. These are specific features like "a user can book a scooter" or "a user can see their ride history."
  • Non-Functional Requirements: Define how the system should perform. These include qualities like scalability (handling 100k requests), availability (99.9% uptime), reliability, and security.

Explain Vertical vs. Horizontal Scaling.​

Answer:

  • Vertical Scaling (Scaling Up): Increasing the capacity of a single server (e.g., adding more CPU, RAM, or faster SSDs). It is simple but limited by the physical capacity of a single machine and becomes very expensive at the high end.
  • Horizontal Scaling (Scaling Out): Adding more servers to the pool and distributing the load across them using a Load Balancer. It allows for virtually unlimited growth and provides better fault tolerance, as the system can continue to work if one node fails.

What are the main strategies for scaling a database?​

Answer:

  • Replication: Creating copies of the database (Master/Slave). Usually, the Master handles writes, while Slaves handle read queries to distribute the load.
  • Sharding: Distributing data across multiple independent database servers (shards) based on a specific key (e.g., User ID, Geolocation). Each shard holds a subset of the total data.
  • Partitioning: Splitting a large table into smaller, more manageable logical pieces within a single database instance to improve query performance.

What is a Sharding Key and how do you choose a good one?​

Answer: A sharding key is the attribute used to determine which shard a specific piece of data belongs to. A good sharding key ensures:

  • Even Distribution: Data is spread uniformly across shards, avoiding "hot spots."
  • Query Efficiency: It minimizes the need for "cross-shard" queries by keeping related data together. Common keys include User ID, Tenant ID, or a Hash of a unique attribute.

What are the best practices for designing a REST API?​

Answer: Use plural nouns for resources (e.g., /users), standard HTTP methods (GET, POST, PUT, DELETE), versioning (e.g., /v1/...), and proper HTTP status codes to indicate the outcome of the request.

What is the relationship between Latency, Throughput, and Concurrency?​

Answer: They are related by Little's Law: Concurrency = Throughput × Latency.

  • Latency: The time it takes to process a single request.
  • Throughput: The number of requests the system can handle per unit of time.
  • Concurrency: The number of requests being processed simultaneously. If you increase the number of parallel requests (concurrency) without increasing the system's throughput, the latency will automatically increase, leading to a bottleneck.

What is a Load Balancer and what are the common balancing algorithms?​

Answer: A Load Balancer (e.g., Nginx, HAProxy, AWS ELB) is a component that distributes incoming network traffic across multiple servers to ensure no single server becomes overwhelmed. Common algorithms include:

  • Round Robin: Requests are distributed sequentially.
  • Least Connections: Sends traffic to the server with the fewest active connections.
  • IP Hash: The client's IP address determines which server receives the request (useful for session persistence).
  • Latency-Aware: The balancer tracks historical response times and avoids "slow" nodes.

What are Circuit Breaker and Back Pressure patterns?​

Answer:

  • Circuit Breaker: Monitors calls to a failing service. If the error rate exceeds a threshold, the "circuit opens," and the system stops calling the service, returning a fast fallback error instead of waiting for timeouts. This prevents cascading failures.
  • Back Pressure: A mechanism where a system "pushes back" against a high rate of incoming requests that it cannot process. Instead of letting queues grow indefinitely (which increases latency), it rejects new requests (e.g., HTTP 429) or asks the client to slow down.

What is Idempotency in API design and why is it important?​

Answer: An idempotent operation is one that has the same effect regardless of how many times it is executed. For example, PUT and DELETE should be idempotent, while POST is typically not. In distributed systems, idempotency is critical for handling network failures. If a client doesn't receive a confirmation due to a timeout, it might retry the request. If the API is idempotent, the retry won't cause unintended side effects (like charging a customer twice).

Senior​

Compare 2PC, 3PC, TCC, and the Saga Pattern.​

Answer:

  • 2PC (Two-Phase Commit): A synchronous protocol for distributed transactions. It ensures strict consistency but is blocking and can be a performance bottleneck.
  • 3PC (Three-Phase Commit): An extension of 2PC that adds a "Pre-commit" phase to reduce blocking in case of coordinator failure. It is rarely used in production.
  • TCC (Try-Confirm-Cancel): An application-level pattern. You reserve resources (Try), and then either finalize (Confirm) or release (Cancel). It is flexible but complex to implement.
  • Saga Pattern: A sequence of local transactions where each step has a corresponding "compensating transaction" to undo changes if a failure occurs. It is highly scalable and common in microservices but offers only eventual consistency.

How can you reduce Tail Latency in a distributed system?​

Answer: Tail latency (P99+) can be reduced using several techniques:

  • Hedged Requests (Tied Requests): Sending the same request to multiple replicas and using the result from the first one that responds.
  • Request Coalescing: Combining identical concurrent requests into a single backend call.
  • Background Jittering: Adding random delays to scheduled tasks to prevent "thundering herd" problems.
  • Deadline Propagation: Passing the maximum allowed time for a request to all sub-services so they can abort early if the deadline is already passed.
  • Latency-Aware Balancing: Avoiding nodes that are currently performing poorly.

Compare different Sharding routing strategies: Application-level, Proxy, and Coordinator.​

Answer:

  • Application-Level Routing: The application code contains the logic to determine which shard to use. It's fast (no extra network hop) but hard to maintain as the sharding logic is duplicated across services.
  • Proxy-Based Routing: A separate layer (e.g., Vitess, ProxySQL) sits between the app and the database. It handles routing, making sharding transparent to the app. It adds a small network hop but centralizes management.
  • Coordinator-Based: A separate service (like Zookeeper or a custom metadata service) keeps track of shard locations. Clients query the coordinator to find the correct shard. It's robust but complex to implement.

Explain the difference between Eventual and Strong Consistency.​

Answer:

  • Eventual Consistency: Replicas will eventually converge to the same value, but there might be a delay where different users see different data. It provides high availability and low latency. Suitable for comments, likes, or product catalogs.
  • Strong Consistency: Guarantees that once a write is successful, any subsequent read will return the new value. It is essential for financial transactions, inventory management, or booking systems.
  • Linearizability: A stronger form of consistency where operations appear to happen instantaneously and in the order they were issued.

What is Tail Latency and why does it matter in distributed systems?​

Answer: Tail latency refers to the high latency experienced by a small percentage of requests (e.g., the 99.9th percentile). In a large-scale system where one user request might trigger hundreds of sub-requests to various microservices, a high tail latency in even one service can slow down the entire response for the user, making it a critical metric for system reliability.

Detailed System Design Principles Guide