Secret Detector: Preventing Secrets in Source Code at Yelp

Ideas you will meet

Distributed Systems

Consensus, time, failure and the fundamentals of scale-out.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Core Concepts: How Things Actually Work · 9 minSSO, OAuth 2.0 and OpenID ConnectHow 'Sign in with Google' works, and the difference between authentication and authorisation.Read hereClose

  Two questions, two words

  • Authentication: who are you?
  • Authorisation: what are you allowed to do?

  OAuth 2.0: delegated access

  OAuth 2.0 lets a user give an app limited access to their data at another service, without sharing their password. Roles:

  • Resource owner: the user.
  • Client: the app asking for access.
  • Authorization server: issues tokens (for example Google, GitHub).
  • Resource server: the API holding the data.

  The user grants specific scopes (for example "read my calendar"), and the client receives an access token that is limited to those scopes and expires.

  The authorization code flow

  100%

  Figure SDA-CORE-07-1 · Authorization code flow · © forcsguy.com

  Important details:

  • state: a random value you check on return, which stops attackers forging the callback (CSRF).
  • PKCE: a one-time proof that the party exchanging the code is the one that started the flow. Use it for all clients, essential for mobile and single-page apps.
  • Register exact redirect URIs and compare them strictly.

  OpenID Connect: adding identity

  OAuth alone says "this token may call the API", not "this is Jane". OpenID Connect (OIDC) adds an ID token (a signed JWT) with claims about the user: subject id, email, name. To sign someone in, validate the ID token's signature, issuer, audience and expiry.

  SSO

  With single sign-on, many applications trust one identity provider. The user signs in once; the provider vouches for them to each app. Enterprises use SAML or OIDC for this. Benefits: fewer passwords, central policies like MFA, and one place to revoke access when someone leaves.

  Single sign-on means the second application never sees your password. It trusts the sign-in service you already used:

  100%

  Figure SDA-CORE-07-2 · One sign-in, two applications · © forcsguy.com

  Common mistakes

  • Using the access token as proof of identity instead of validating an ID token.
  • Skipping state or accepting any redirect URI.
  • Storing tokens somewhere scripts can read them.
  • Requesting more scopes than the feature needs.

  More on this site

  Related reading

  • RFC 6749: The OAuth 2.0 Authorization FrameworkIETFDocumentation
  • OAuth 2.0 OverviewOAuth Community · oauth.netDocumentation
  • Introduction to JSON Web TokensAuth0 · jwt.ioDocumentation
  • System Design 101ByteByteGo · ByteByteGoHqGitHub

  Show 6 moreShow fewer

  • OAuth Audit Toolbox at IndeedIndeed engineers · IndeedCase Studies
  • The Architecture of Open Source ApplicationsVarious authors · aosabook.orgBooks
  • Secret Detector: Preventing Secrets in Source Code at YelpYelp engineers · YelpCase Studies
  • LevelDBGoogle · GitHubGitHub
  • Garbage Collection in Go Application at TwitchTwitch engineers · TwitchCase Studies
  • DevOps resource center - Azure DevOpsMicrosoft engineers · MicrosoftBlogs

  Open this lesson with its quiz and exercise

• Real-World Systems · 11 minDesign an AI coding assistantHow an editor assistant suggests code in under a second: gathering context, retrieval, the model, streaming and keeping private code safe.Read hereClose

  The problem

  An assistant inside the editor that completes code as you type, answers questions in a chat panel, and can edit files. The hard constraints:

  • Latency: inline suggestions must appear in roughly a few hundred milliseconds, or the user has already moved on.
  • Relevance: good suggestions need the right context from a large codebase, not just the current line.
  • Cost: language-model inference is expensive per request, and typing generates a request storm.
  • Privacy and security: customer source code and secrets are sensitive.
  • Quality: wrong or insecure suggestions must be limited and measurable.

  Architecture

  100%

  Figure SDA-RWS-18-1 · AI coding assistant architecture · © forcsguy.com

  1. The editor extension

  Runs in the user's editor. It watches typing, debounces requests (waits a few tens of milliseconds), cancels outdated requests when the user continues typing, and streams the suggestion into the editor as ghost text. It also handles accept, reject and partial accept, and enforces local privacy rules (excluded files, secret detection).

  2. Building context

  Prompt = instructions + context + the code around the cursor. Context comes from:

  • The current file: the code before the cursor (prefix) and after it (suffix, for fill-in-the-middle).
  • Open tabs and recently edited files.
  • Imports and related files: definitions of functions and types being used.
  • Retrieved snippets from the wider project.

  All of it must fit the model's context window and a token budget, so the builder ranks pieces by relevance and trims.

  3. Retrieval over a codebase

  A repository is too big to include whole, so it is indexed:

  1. Split files into chunks (functions, classes, blocks).
  2. Compute embeddings (vectors capturing meaning) and store them in a vector index, together with a symbol and dependency graph (what calls what).
  3. At request time, search with the current code as the query and combine semantic results with structural ones (definitions, callers, tests).
  4. Update the index incrementally as files change, and keep per-user and per-organisation isolation.

  4. Model serving

  • A language model generates tokens. Serving systems batch many requests together, reuse a KV cache of already-processed prompt tokens, and stream output so the first words show immediately.
  • Use smaller, faster models for inline completion and larger ones for chat and complex edits.
  • Cache repeated prompts and completions where context is identical, and prefix-cache shared prompt parts.
  • Autoscale GPU capacity, and route by region for latency.

  5. Post-processing and safety

  • Trim the output to a sensible stopping point (end of block or line).
  • Filter secrets, personal data and known insecure patterns; optionally check for verbatim public code matches and licence concerns.
  • Reject low-confidence suggestions instead of showing noise.
  • For agents that edit files or run commands, require explicit user approval and run actions in a sandbox.

  100%

  Figure SDA-RWS-18-2 · How a code model is trained and judged · © forcsguy.com

  Latency budget

  For inline suggestions, a budget like this keeps the experience smooth:

  Step	Target
  Debounce and local context	about 30 ms
  Network to the service	about 30 ms
  Retrieval and prompt building	about 30 ms
  Model: time to first token	about 150 ms
  Streaming and display	continuous

  Everything else (retrieval, indexing, safety checks) must fit inside it or run asynchronously ahead of time.

  Privacy, security and trust

  • Send only what is needed, respect ignore files, and redact secrets.
  • Provide enterprise controls: private deployment or isolated tenancy, no training on customer code, short retention, audit logs.
  • Encrypt in transit and at rest; strong authentication and per-user quotas.
  • Defend against prompt injection (malicious text in files or web pages trying to instruct the model): treat retrieved content as data, limit tool permissions, require confirmation for risky actions.

  Measuring quality

  • Offline evaluation: run the model on held-out code and tests, measure pass rates and exact-match similarity.
  • Online metrics: acceptance rate of suggestions, characters kept after acceptance, latency percentiles, and errors.
  • A/B test changes to prompts, models and retrieval, and watch for regressions in security and licence filters.
  • Collect feedback signals (thumbs up or down) while respecting privacy.

  What this design reuses

  Debouncing and cancellation, caching, rate limiting and quotas, queues and batching, vector search, streaming responses and careful privacy controls: the same building blocks from the rest of this course, applied to a latency-sensitive AI workload.

  Real-world reading

  Related reading

  • Secret Detector: Preventing Secrets in Source Code at YelpYelp engineers · YelpCase Studies
  • Samza: Stream Processing System for Latency Insighs at LinkedInLinkedIn engineers · LinkedInCase Studies
  • Use The Index, LukeMarkus Winand · use-the-index-luke.comCourses
  • Backend Developer Roadmaproadmap.sh community · roadmap.shCourses

  Show 6 moreShow fewer

  • Natural Language Processing and Content Analysis (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • Machine Learning for Indexing Text from Billions of Images at DropboxDropbox engineers · DropboxCase Studies
  • Optimizing Web Servers for High Throughput and Low Latency at DropboxDropbox engineers · DropboxCase Studies
  • Scalable User Privacy at SpotifySpotify engineers · SpotifyCase Studies
  • Privacy-Preserving Analytics and Reporting at LinkedInLinkedIn engineers · LinkedInCase Studies
  • Learning with Privacy at Scale at AppleScale at Apple engineers · Scale at AppleCase Studies

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 10 minATM machineA machine that behaves differently in each state: the State pattern, card and PIN checks, and cash dispensing with limits.Read hereClose

  The problem

  An ATM accepts a card, checks the PIN, then lets the customer check balance, withdraw cash or deposit, and returns the card. It talks to a bank to authorise operations and holds a limited amount of cash.

  States

  100%

  Figure SDA-LLD-06-1 · ATM states · © forcsguy.com

  In Idle, "enter PIN" makes no sense; in Authenticated, "insert card" makes no sense. The State pattern puts those rules inside state objects instead of if (state === ...) chains.

  interface AtmState {
    insertCard(atm: Atm, card: Card): void;
    enterPin(atm: Atm, pin: string): void;
    withdraw(atm: Atm, cents: number): void;
  }
  
  class Idle implements AtmState {
    insertCard(atm, card) { atm.card = card; atm.setState(new HasCard()); }
    enterPin() { throw new Error("insert a card first"); }
    withdraw() { throw new Error("insert a card first"); }
  }
  class HasCard implements AtmState {
    tries = 0;
    insertCard() { throw new Error("card already inserted"); }
    enterPin(atm, pin) {
      if (atm.bank.checkPin(atm.card!, pin)) atm.setState(new Authenticated());
      else if (++this.tries >= 3) { atm.retainOrEject(); atm.setState(new Idle()); }
    }
    withdraw() { throw new Error("enter the PIN first"); }
  }
  

  Who is responsible for what

  100%

  Figure SDA-LLD-06-2 · Withdrawal sequence · © forcsguy.com

  • Bank owns the balance and account rules. ATM owns hardware, cash on hand and the session.
  • Cash dispenser holds counts per denomination and decides which notes to give (greedy from the largest, with a fallback).
  • Every step is logged so a failure between "dispensed" and "confirmed" can be reconciled later (a mismatch report, or an automatic reversal).

  Design notes

  • Treat PINs as secrets: never store or log them; the bank verifies, the ATM only forwards an encrypted PIN block.
  • Idempotency: a retried authorise call must not debit twice; use a transaction id.
  • Timeouts: an idle session returns to Idle and ejects the card.
  • Extensions: deposits (count and hold until verified), fund transfers, receipts, and multiple card types.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Optimizing Payments with Machine Learning at DropboxDropbox engineers · DropboxCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies

  Show 6 moreShow fewer

  • PyKrylov: Accelerating Machine Learning Research at eBayeBay engineers · eBayCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • Machine Learning Platform at ZalandoZalando engineers · ZalandoCase Studies
  • Secret Detector: Preventing Secrets in Source Code at YelpYelp engineers · YelpCase Studies
  • Machine Learning at Jane StreetJane Street engineers · Jane StreetCase Studies
  • Waymo BlogWaymo engineers · WaymoBlogs

  Open this lesson with its quiz and exercise