In the previous article, I compared cloud and local models for the same seller listing quality task. The conclusion was not that one model wins every time. It was that we should choose a model according to the requirements of the feature.
Model choice is still only one part of a production AI feature.
A listing review can contain suggestions. A catalog enricher is different. If the API proposes a publisher, ISBN, author, publication date, or missing description, I want to know where that value came from. If it infers a theme, audience, or reading difficulty, I want the API to say clearly that this is an interpretation rather than a catalog fact. I also do not want a temporary quota problem in the primary cloud model to make the entire feature unavailable.
In this article, we will add a second Spring Boot application to the project. It will:
- enrich book listings with evidence from Google Books
- derive useful discovery metadata from cited seller and catalog evidence
- use Gemini 3.5 Flash as the primary model
- fall back to a configurable local Gemma 4 model through oMLX
- reject proposals that cannot be verified against seller input or catalog evidence
- return proposals for seller approval, never apply them automatically
This continues the same project from the first three articles:
TL;DR – SmtC
Too Long; Didn’t Read – Show me the Code: https://github.com/iseif/listing-quality
Why a separate Maven module?
The existing service reviews a listing. The new service enriches catalog data. They have different APIs, dependencies, failure policies, and deployment characteristics, so I added a sibling module instead of putting everything into one application:
listing-quality/
├── listing-quality-service/
├── listing-quality-enrichment/
└── listing-quality-evaluation/
The packages inside the new module also leave room for other catalog types:
catalog.book
catalog.book.google
model.book
service.book
service.execution
catalog.book owns the provider-neutral book catalog contract. catalog.book.google owns the Google Books transport. If we add another book catalog later, it belongs beside google. model.book and service.book contain the book-specific contract and workflow.
Retry, timeout, circuit-breaker, and provider failure classes are not book concepts, so they live in service.execution. A future apparel enricher can reuse that infrastructure. I kept BookEnrichmentGenerator book-specific instead of introducing a generic EnrichmentGenerator<TRequest, TResponse> before a second implementation needs that abstraction.
The API contract
The endpoint accepts seller input:
POST /api/enrichments/books
X-API-Version: 1
Content-Type: application/json
{
"listingId": "book-123",
"title": "Effective Java by Joshua Bloch",
"description": "",
"price": 24.99,
"attributes": {
"condition": "used",
"isbn13": "9780134686042"
}
}
The response separates five concepts:
proposals: factual catalog values that are absent from the seller listingconflicts: catalog values that disagree with existing seller valuesderivedAttributes: bounded interpretations derived from cited evidenceunresolvedFields: fields that could not be resolved safelywarnings: partial failures such as a catalog lookup being unavailable or an unverifiable model item being rejected
It also returns execution.route, which is either PRIMARY or FALLBACK, and requiresSellerApproval.
Separate catalog facts from discovery metadata
The first version of the contract only enriched factual catalog fields. That is safe, but a shopping assistant also needs metadata such as genre, themes, tone, pacing, target audience, reading difficulty, keywords, and a concise summary.
Those values cannot use the same validation rule as a publisher or ISBN. A catalog publisher proposal must equal the cited publisher field. A theme such as software craftsmanship is inferred from a description; it should not be expected to equal the description.
The response therefore has a separate type:
public record DerivedBookAttribute(
BookDiscoveryField field,
JsonNode value,
DerivationType derivation,
List<Evidence> evidence) {
}
derivation is currently INFERRED_FROM_EVIDENCE. The supported discovery fields are genres, target audience, minimum and maximum age, tone, themes, pacing, reading difficulty, book length estimate, keywords, and short summary.
The values are bounded by the application. Lists contain at most six short values, enum-like fields use a fixed vocabulary, ages must be between 0 and 120 with minAge <= maxAge, and the summary is limited to 300 characters. A short summary must cite a seller or catalog description.
The catalog description itself remains a factual field. The API can propose it only when the seller description is blank and the proposed text exactly matches the cited Google Books description. It never replaces a nonblank seller description.
I intentionally left out estimated_rating, popularity_score, page_turner_score, and similar numeric scores. Seller input and Google Books metadata cannot verify them. Adding precise-looking numbers would make the response richer in shape but weaker in trust.
Give the model controlled tools, not unrestricted internet access
The model sometimes needs external information, but that does not mean it should receive an arbitrary web browser or choose its own URL.
For this service, the model receives two request-scoped tools:
@Tool(description = "Find one book by a seller-provided ISBN. The application validates the ISBN and uses a fixed Google Books endpoint.")
public CatalogLookupResult findBookByIsbn(
@ToolParam(description = "ISBN-10 or ISBN-13, with or without hyphens") String isbn) {
return Isbn.normalize(isbn)
.map(this::lookupIsbn)
.orElseGet(() -> CatalogLookupResult.empty(CatalogLookupStatus.INVALID_QUERY));
}
@Tool(description = "Search for up to three books by seller-provided title and author. Ambiguous results cannot support factual proposals.")
public CatalogLookupResult searchBooks(
@ToolParam(description = "Book title from the seller listing") String title,
@ToolParam(description = "Author from the seller listing, or an empty string") String author) {
String normalizedTitle = normalizeText(title);
if (normalizedTitle.isBlank()) {
return CatalogLookupResult.empty(CatalogLookupStatus.INVALID_QUERY);
}
String normalizedAuthor = normalizePerson(author);
String key = "title:" + normalizedTitle + "|author:" + normalizedAuthor;
return ledger.cached(key).orElseGet(() -> searchAndRecord(
key, title.trim(), author == null ? "" : author.trim(), normalizedTitle, normalizedAuthor));
}
The application, not the model, controls the Google Books base URL. The adapter also bounds the timeout, response size, number of results, and retry policy. ISBN values are checksum validated before a lookup.
Google Books occasionally returns a transient 503, so the adapter retries the idempotent GET request for transport failures and the HTTP statuses 408, 429, 500, 502, 503, and 504. It does not retry permanent client failures such as 400, 401, 403, or 404. The default is three total attempts with bounded exponential backoff and jitter:
listing-quality:
enrichment:
catalog:
google-books:
max-attempts: ${GOOGLE_BOOKS_MAX_ATTEMPTS:3}
initial-backoff: ${GOOGLE_BOOKS_INITIAL_BACKOFF:250ms}
max-backoff: ${GOOGLE_BOOKS_MAX_BACKOFF:1s}
The request-scoped cache records only the final lookup result. A transient first response therefore does not become a cached catalog miss before the adapter has exhausted its retry policy. Retry logs contain only a failure category, attempt number, and delay; they do not contain the API key, seller input, request URI, or provider response body.
Each request creates a BookCatalogEvidenceLedger. It records the exact catalog records returned to the model, their match type, and catalog warnings. It also caches repeated lookups within that request.
This ledger is important. Later, when the model returns a proposed publisher and cites record abc123, the validator can prove that abc123 was actually returned during this request and that its publisher equals the proposed value.
Seller text and catalog text are both untrusted data. An instruction-like book title is data, not a new system instruction. The user prompt serializes the complete listing as JSON inside a resource template, while the system prompt defines the rules.
Two ChatClients, one generator implementation
The application uses the Google GenAI Spring AI starter for Gemini. oMLX exposes an OpenAI-compatible API, so the second client uses OpenAiChatModel with a local base URL.
Both clients are built through ChatClientBuilderConfigurer:
private ChatClient buildChatClient(
ChatModel chatModel,
ChatClientBuilderConfigurer configurer,
ObjectProvider<ObservationRegistry> observationRegistry,
ObjectProvider<ChatClientObservationConvention> chatObservationConvention,
ObjectProvider<AdvisorObservationConvention> advisorObservationConvention,
ObjectProvider<ToolCallingAdvisor.Builder<?>> toolCallingAdvisorBuilder,
String systemPrompt) {
ChatClient.Builder builder = ChatClient.builder(
chatModel,
observationRegistry.getIfUnique(() -> ObservationRegistry.NOOP),
chatObservationConvention.getIfUnique(),
advisorObservationConvention.getIfUnique(),
toolCallingAdvisorBuilder.getIfAvailable());
return configurer.configure(builder)
.defaultSystem(systemPrompt)
.build();
}
Using the configurer keeps Spring AI customizers, advisors, tool execution, and observations active for both clients.
The oMLX model is constructed inside the oMLX ChatClient bean. It is not published as a second ChatModel bean. This avoids the ambiguous ChatModel startup failure we saw when multiple model starters were active.
Both clients use the same adapter:
public final class SpringAiBookEnrichmentGenerator implements BookEnrichmentGenerator {
private final String providerId;
private final ChatClient chatClient;
private final StructuredOutputValidationAdvisor validationAdvisor;
public SpringAiBookEnrichmentGenerator(
String providerId,
ChatClient chatClient,
int structuredOutputAttempts) {
this.providerId = Objects.requireNonNull(providerId);
this.chatClient = Objects.requireNonNull(chatClient);
if (structuredOutputAttempts < 1) {
throw new IllegalArgumentException(
"structuredOutputAttempts must be at least one");
}
this.validationAdvisor = StructuredOutputValidationAdvisor.builder()
.outputType(GeneratedBookEnrichment.class)
.maxRepeatAttempts(structuredOutputAttempts - 1)
.build();
}
@Override
public GeneratedBookEnrichment generate(String prompt, BookCatalogTools tools) {
try {
return chatClient.prompt()
.user(prompt)
.tools(tools)
.advisors(validationAdvisor)
.call()
.entity(GeneratedBookEnrichment.class);
} catch (TransientAiException exception) {
throw ModelExecutionException.eligible(
providerId, ModelFailureCategory.PROVIDER_UNAVAILABLE, exception);
} catch (NonTransientAiException exception) {
throw ModelExecutionException.ineligible(
providerId, ModelFailureCategory.CONFIGURATION_ERROR, exception);
} catch (RuntimeException exception) {
throw ModelExecutionException.eligible(
providerId, ModelFailureCategory.INVALID_MODEL_OUTPUT, exception);
}
}
}
I configure the validation advisor explicitly because retry ownership matters. In Spring AI 2.0, entity(type, spec -> spec.validateSchema()) creates an advisor with three repeat attempts by default. Wrapping that call in another retry loop would multiply model calls and could consume the route timeout before fallback starts. Here, structuredOutputAttempts means the total number of model calls for schema correction, so a value of 2 becomes one initial call and one repeat.
One structured-output detail is easy to miss here. The model-facing DTO should not use JsonNode for a value that may be a string, list, or integer. Spring AI generates JSON Schema type object for JsonNode, so a provider that returns a valid string or array will fail schema validation before the application validator sees it.
I use an explicit transport value instead:
public record GeneratedBookValue(
GeneratedBookValueKind kind,
String text,
List<String> items,
Integer integer) {
}
Spring AI marks record components as required in the generated schema, so the model returns all four properties. kind selects the active slot and the other slots use neutral values: an empty string, an empty list, or zero. The validator rejects a value whose populated slot does not match kind, then converts the selected value to the natural JSON type used by the public API. This keeps the provider schema precise without leaking the transport wrapper into the seller-facing response.
Changing the route needs configuration only:
spring:
ai:
google:
genai:
chat:
model: ${GEMINI_CHAT_MODEL:gemini-3.5-flash}
max-output-tokens: ${GEMINI_MAX_OUTPUT_TOKENS:8192}
thinking-level: MINIMAL
listing-quality:
enrichment:
primary: ${ENRICHMENT_PRIMARY:gemini}
fallback: ${ENRICHMENT_FALLBACK:omlx}
providers:
gemini:
model: ${GEMINI_CHAT_MODEL:gemini-3.5-flash}
omlx:
base-url: ${OMLX_BASE_URL:http://localhost:8000/v1}
model: ${OMLX_CHAT_MODEL:mlx-community/gemma-4-e4b-it-4bit}
The output budget is intentionally larger than in the listing-quality API. An enrichment response may contain a description, multiple proposals, inferred attributes, and evidence for each value. If the provider reaches a small output ceiling, the result is an unfinished JSON document, not a schema problem that repeated prompts can repair. The environment variable keeps the limit adjustable for a different model or contract.
Choose a local model for the workflow, not only the hardware
The configuration above uses Gemma 4 E4B as an accessible starting point. It fits on many recent machines, but fitting in memory is not the same as completing this workflow reliably.
This request combines several model capabilities. The model must choose and call a catalog tool, interpret its result, return a large structured response, and cite exact record IDs and source fields. During my live checks, Gemma 4 E4B ran through oMLX but repeatedly returned evidence references that the application could not verify. In one response, TARGET_AUDIENCE and BOOK_LENGTH_ESTIMATE cited invalid evidence. The validator quarantined both attributes and returned a safe NEEDS_SELLER_INPUT response with UNVERIFIABLE_MODEL_ITEM_REJECTED, but the response contained no useful enrichment.
I then downloaded gemma-4-26B-A4B-it-MLX-8bit from the oMLX admin dashboard and ran the same workflow again. On my MacBook Pro with an Apple M4 Pro and 48 GB of unified memory, it completed the tested enrichment successfully:
OMLX_CHAT_MODEL=gemma-4-26B-A4B-it-MLX-8bit \
./mvnw -pl listing-quality-enrichment spring-boot:run
| Local model | Observation in this workflow |
|---|---|
| Gemma 4 E4B, 4-bit | Ran locally, but repeatedly produced invalid or unusable grounded items |
| Gemma 4 26B-A4B, 8-bit | Completed the tested enrichment successfully on the M4 Pro with 48 GB |
This is a practical observation, not a general benchmark. It does not prove that the 26B model will succeed for every book or that E4B cannot handle a simpler task. It shows why a local fallback must be qualified against representative application requests. OpenAI-compatible transport and tool support are necessary, but they do not guarantee that every model can follow the same multi-step contract with the same reliability.
There is also a subtle fallback policy here. Invalid individual items are quarantined so that independently valid items can survive. Quarantine returns a successful, degraded route result; it does not throw a route failure. Therefore, if every generated item is rejected, the current implementation returns a safe response instead of calling the fallback model. A production policy could classify zero accepted items as an insufficiently useful result and activate fallback, but that behavior is not implemented in this version.
Validate evidence before accepting the answer
Schema validation proves that the model returned the expected JSON shape. It does not prove that a publisher, ISBN, or author came from an allowed source.
BookEnrichmentValidator performs the second validation layer. It owns the policy and delegates the mechanics: BookEvidenceResolver proves citations, DerivedBookValueValidator bounds the discovery vocabulary, and BookValueMapper reads seller and catalog values. Factual proposals use an equality rule. For every Google Books reference it requires:
- a catalog record ID
- a record that exists in the request ledger
- an allowed source field
- a proposed value equal to the source value
- a safe match type, such as exact ISBN or one exact title and author result
If the seller already supplied a different factual value, the validator converts the model proposal into a conflict. Description is stricter: it can only fill a blank value, never replace seller text.
Derived attributes use a different rule. The validator still requires every cited record to exist in the request ledger and every cited field to contain a value, but it does not compare the inferred value for equality with the evidence. It validates the field-specific vocabulary, bounds, and evidence allowlist instead. A publisher cannot be used as evidence for a theme, for example. This keeps facts and interpretations honest without pretending that a theme must literally appear in the source text.
Evidence and derived-value validation are scoped to the individual item. If a proposal cites a missing record, uses a disallowed source field, does not equal its cited factual value, or a derived value falls outside its allowed vocabulary or bounds, the application quarantines that item. A rejected factual field is added to unresolvedFields, the response includes UNVERIFIABLE_MODEL_ITEM_REJECTED, and independently validated proposals and derived attributes remain available. The rejected value never crosses the API boundary.
This is deliberately narrower than ignoring arbitrary validation failures. Structural contract failures, duplicate fields, malformed value objects, invalid ISBNs, description-overwrite attempts, and inconsistent age ranges still reject the complete route and can activate fallback. This gives us graceful degradation for independently verifiable fields without weakening the trust boundary.
What the API actually returns
This is a real response to the request shown earlier, where the seller supplied a title and an ISBN but left the description blank. The long catalog description is shortened for readability:
{
"status": "PROPOSALS_AVAILABLE",
"proposals": [
{
"field": "AUTHORS",
"currentValue": null,
"proposedValue": ["Joshua Bloch"],
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "authors",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "PUBLISHER",
"currentValue": null,
"proposedValue": "Addison-Wesley Professional",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "publisher",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "PUBLISHED_DATE",
"currentValue": null,
"proposedValue": "2017-12-18",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "publishedDate",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "DESCRIPTION",
"currentValue": null,
"proposedValue": "The Definitive Guide to Java Platform Best Practices [shortened for this article]",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "description",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "ISBN10",
"currentValue": null,
"proposedValue": "0134686047",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "isbn10",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "LANGUAGE",
"currentValue": null,
"proposedValue": "en",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "language",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "PAGE_COUNT",
"currentValue": null,
"proposedValue": 957,
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "pageCount",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "CATEGORIES",
"currentValue": null,
"proposedValue": ["Computers"],
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "categories",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
}
],
"conflicts": [],
"derivedAttributes": [
{
"field": "GENRES",
"value": ["TECHNOLOGY"],
"derivation": "INFERRED_FROM_EVIDENCE",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "categories",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "TARGET_AUDIENCE",
"value": "ADULT",
"derivation": "INFERRED_FROM_EVIDENCE",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "description",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "BOOK_LENGTH_ESTIMATE",
"value": "LONG",
"derivation": "INFERRED_FROM_EVIDENCE",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "pageCount",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "SHORT_SUMMARY",
"value": "A thoroughly updated guide to Java platform best practices, covering language and library features added in Java 7, 8, and 9.",
"derivation": "INFERRED_FROM_EVIDENCE",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "description",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
},
{
"field": "KEYWORDS",
"value": ["Java", "Programming", "Best Practices", "Software Development"],
"derivation": "INFERRED_FROM_EVIDENCE",
"evidence": [
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "title",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
},
{
"source": "GOOGLE_BOOKS",
"recordId": "BIpDDwAAQBAJ",
"sourceField": "description",
"sourceUrl": "https://www.googleapis.com/books/v1/volumes/BIpDDwAAQBAJ",
"matchType": "EXACT_ISBN"
}
]
}
],
"unresolvedFields": ["SUBTITLE", "TITLE"],
"warnings": ["UNVERIFIABLE_MODEL_ITEM_REJECTED"],
"requiresSellerApproval": true,
"execution": {
"route": "PRIMARY"
}
}Several parts of this response are worth reading closely.
Every factual proposal carries a record ID, the exact catalog field it came from, a resolvable source URL, and EXACT_ISBN. The seller supplied a valid ISBN-13, so the strongest match type was available and the equality rule had something precise to check against. The seller left the description blank, so the API was allowed to propose the catalog description; had the seller written anything there, that proposal would have been rejected rather than turned into a conflict.
The derived attributes look similar but claim something different. BOOK_LENGTH_ESTIMATE is LONG because the record reports 957 pages, and SHORT_SUMMARY cites the catalog description. Neither value appears literally in its evidence, and neither is presented as a catalog fact. GENRES is constrained to the application’s vocabulary, which is why a Google Books category of Computers becomes TECHNOLOGY rather than free text.
The most useful part is what the response does not contain. TITLE and SUBTITLE are absent from proposals and present in unresolvedFields, and they got there by two different routes.
TITLE was rejected. The application log for this request contains exactly one quarantine line:
Rejected unverifiable enrichment item: failure=EVIDENCE_VALUE_MISMATCH, field=TITLEThe model proposed a title, cited the catalog title as its evidence, and the proposed value did not equal that cited field. So the validator quarantined the item, added TITLE to unresolvedFields, and raised UNVERIFIABLE_MODEL_ITEM_REJECTED. The rejected value never reached the response.
Note what this is not. It is not a conflict. A conflict requires a proposal that first survives evidence validation and then disagrees with existing seller data. This proposal never earned that status, which is why conflicts is empty even though the seller title and the catalog title differ.
SUBTITLE was never rejected. No log line names it, so it came from the model’s own unresolvedFields output: this edition simply has no subtitle to propose. One field failed verification and one was honestly declared unresolvable, and both end up in the same list, which is what the seller needs.
Nothing was silently corrected and nothing was silently dropped. The response states which fields it could not resolve, and requiresSellerApproval is true, so a human still decides.
Retry, timeout, circuit breaker, and fallback are different
These mechanisms solve different problems:
- structured-output correction retries malformed JSON from the same model
- catalog transport retry repeats safe Google Books lookups after short-lived connection, throttling, or server failures
- timeout bounds one provider route
- circuit breaker stops calling a provider that is repeatedly failing
- fallback makes one attempt through a different provider route
The service falls back for provider unavailability, rate limits, exhausted quota, timeouts, an open circuit, invalid structured output, or a validation failure that cannot be safely quarantined.
It does not fall back for authentication errors, invalid configuration, invalid client input, or a safety refusal. Sending the same bad credentials to another route hides an operational problem instead of fixing it.
There is also one overall request deadline. The fallback receives only the remaining time, so a slow primary cannot cause an unbounded second call.
Concise errors without leaking provider data
Provider messages can contain request fragments, seller data, or account details. The domain exception stores a provider ID, failure category, and fallback eligibility, but its public message is only:
AI model execution failed
The HTTP layer returns RFC 9457 Problem Details:
400 BAD_REQUESTfor invalid seller input502 ENRICHMENT_RESPONSE_INVALIDwhen both reachable paths cannot produce grounded output503 ENRICHMENT_PROVIDER_UNAVAILABLEfor a terminal availability failure500 ENRICHMENT_CONFIGURATION_ERRORfor authentication or configuration problems
Tests assert that raw provider messages, secrets, seller text, prompts, and stack traces are absent from the response.
Test the policy without calling paid models
The automated suite is deterministic and offline. It covers ISBN checks, request caching, ambiguous title matches, a bounded local HTTP server for the Google Books adapter, prompt injection boundaries, evidence validation, both ChatClient beans, retry classification, circuit opening, fallback routing, and the HTTP error contract.
At the time of this draft, the module contains 73 passing tests. No test calls Gemini, oMLX, or the real Google Books service. The suite also verifies real Gemini auto-configuration at startup, the generated structured-output schema, the configured output budget, bounded schema-regeneration attempts, transient catalog recovery, exhausted catalog retries, permanent catalog failures without retry, field-level quarantine, safe validation diagnostics, description handling, bounded discovery values, evidence citations, and the separation between factual proposals and derived attributes.
One fallback test forces an eligible quota failure from the primary generator and verifies that the fallback receives the exact same rendered prompt and the same request-scoped tool instance:
given(primaryGenerator.generate("same prompt", tools)).willThrow(
ModelExecutionException.eligible(
"gemini", ModelFailureCategory.QUOTA_EXHAUSTED, new RuntimeException("detail")));
given(fallbackGenerator.generate("same prompt", tools)).willReturn(generated);
BookEnrichmentExecution result = failover.execute(request, "same prompt", tools, ledger);
assertThat(result.route()).isEqualTo(ExecutionRoute.FALLBACK);
verify(fallbackGenerator).generate("same prompt", tools);
Real-provider checks remain a separate release gate. They verify the exact Gemini and Gemma model versions, tool calling, structured output, and actual latency without making the normal build non-deterministic.
Limitations
Google Books is one catalog source, not universal truth. An exact ISBN is stronger than a title search. A title and author search is accepted only when it produces one exact result; ambiguous results remain unresolved. Discovery metadata is still an inference even when it cites a strong catalog match.
The service does not write to eBay, Amazon, Shopify, or any other marketplace. It returns evidence-backed proposals for seller review.
Tool support can also vary between local model artifacts and chat templates. An OpenAI-compatible endpoint does not guarantee identical tool behavior for every model.
The local model observation in this article came from one Apple M4 Pro with 48 GB of unified memory and a limited set of live enrichment requests. It should be treated as workflow qualification, not a universal performance or quality ranking.
What is next?
We now have a grounded enrichment boundary, useful discovery metadata, and a configurable cloud-to-local fallback policy. The next useful step is to apply the same pattern to another catalog concern, such as color normalization or image-derived attributes, and then discuss batching, rate limits, and marketplace integration boundaries.
The important lesson is not simply that a local model can replace a cloud model. It is that either model must earn the right to propose a catalog fact through evidence that the application can verify, and must label interpretations as interpretations.
