LLM¶
Every name on this page is importable from sci_etl_core.llm.
Bases: ABC
Contract for a chat-completion backend that answers with a JSON object.
The relevance filter and the entity extractor depend only on this
interface, so a provider, a cache such as
:class:~sci_etl_core.llm.cache_async.CachingLLMClient, or a test double
can be injected in its place.
property
¶
usage: TokenUsage | None
Tokens this client has used so far, or None when it does not track usage.
AsyncOpenAICompatibleClient(
api_key: str | SecretStr,
base_url: str,
model: str,
default_timeout: int = 120,
temperature: float = 0.0,
max_retries: int = 3,
backoff_factor: float = 2.0,
sleep: Any = sleep,
max_retry_after: float = 60.0,
rate_limiter: RateLimiting | None = None,
)
Bases: AsyncLLMClient
JSON-mode chat completions from any OpenAI-compatible endpoint, with retries and token accounting.
The client owns its HTTP connection pool; close it with :meth:aclose.
Configure the client.
This client is the only retry layer: the OpenAI SDK's own retries are
turned off, so max_retries is the total number of attempts. Between
attempts it waits backoff_factor ** attempt seconds, or longer when
the server's retry-after-ms or Retry-After header asks for it,
up to max_retry_after seconds.
Every attempt first enters rate_limiter, an
:class:~sci_etl_core.rate_limiter.AsyncRateLimiter or a
:class:~sci_etl_core.rate_limiter.HostRateLimiter matched against
base_url, and releases it once the response arrives. Share one
limiter between a chat client and an embedder that call the same
provider to keep both inside one budget.
Raises:
| Type | Description |
|---|---|
ValueError
|
|
property
¶
usage: TokenUsage
Tokens reported across every response received so far, as a snapshot.
classmethod
¶
from_config(
llm: LLMConfig, **options: Any
) -> AsyncOpenAICompatibleClient
Build a client from the llm config section.
The section supplies api_key, base_url, model, and
timeout as default_timeout. options pass any other
constructor argument, such as rate_limiter, and override a value
taken from the config.
async
¶
complete_json(
system_prompt: str,
user_content: str,
timeout: int | None = None,
) -> dict[str, Any]
Request a JSON-mode completion and return the parsed object.
An empty completion reads as {}. Every response the API returns
counts toward :attr:usage, including one whose body is then rejected.
Raises:
| Type | Description |
|---|---|
LLMError
|
The request failed after retries, or the completion is not valid JSON or is JSON other than an object. |
AsyncLLMRelevanceFilter(
llm_client: AsyncLLMClient,
system_prompt: str,
timeout: int = 20,
default_on_empty_abstract: bool = True,
default_on_error: bool = True,
)
Bases: AsyncRelevanceFilter
Ask an LLM whether a record's title and abstract match system_prompt.
Configure the filter.
system_prompt must ask for a JSON object with a relevant key.
timeout is passed to every completion. A record without an abstract
is not sent and reads as default_on_empty_abstract; an
:class:~sci_etl_core.exceptions.LLMError or an unclear verdict reads
as default_on_error. Both default to
True, so a fault costs an extraction call rather than a record.
async
¶
Ask the LLM for a verdict read from the relevant key.
A boolean is used as-is; 0/1 and the strings "true",
"false", "yes", "no", "1" and "0" (any case) are
accepted too. A failed call, a response that is not a JSON object, and a
missing or unrecognized verdict all return default_on_error, so a
garbled answer such as the string "false" read as truthy can never
pass for a confident verdict.
AsyncEmbeddingRelevanceFilter(
embedder: AsyncEmbedder,
reference_texts: Sequence[str],
threshold: float = 0.35,
record_to_text: Callable[
[RawRecord], str
] = _default_record_text,
default_on_empty_abstract: bool = True,
default_on_error: bool = True,
)
Bases: AsyncRelevanceFilter
Keep records whose meaning is close to a set of reference concepts.
Reference concepts are embedded once, lazily, on first use. Each record is
then embedded and kept when its cosine similarity to the nearest reference
reaches threshold, so semantically related work is matched even when it
shares no keywords with the query.
Only a real similarity score can reject a record. A failed embedding call,
a record vector that is missing, all zero or non-finite, and a vector whose
dimension differs from the references all return default_on_error:
none of them is evidence that the record is irrelevant, and a negative
verdict would mark it processed for good.
Bases: ABC
Contract for turning a record's full text into structured entities.
abstractmethod
async
¶
Extract structured entities from raw text.
An empty list means the text holds no entities, and the pipeline marks the record processed. Raising any exception instead fails the record, which stays unmarked and is retried on the next run.
AsyncLLMEntityExtractor(
llm_client: AsyncLLMClient,
system_prompt: str,
html_parser: Parser | None = None,
result_key: str = "items",
max_chars: int = 120000,
timeout: int = 120,
max_tokens: int | None = None,
encoding_name: str = "cl100k_base",
validator: RecordValidator | None = None,
logger: Callable[[str], None] | None = None,
label_field: str | None = None,
)
Bases: AsyncEntityExtractor
Extract entities from full text with one JSON-mode LLM call per record.
Text that starts with markup is stripped with html_parser. The text is
then truncated to max_tokens tokens when that is set and tiktoken
is installed, or to max_chars characters otherwise.
Configure the extractor.
With a validator, each extracted entity is checked before it is
returned, and an entity the validator rejects is dropped and logged as
Entity rejected by validation: <label>. The label is the entity's
label_field value when that is given, and its position in the
response otherwise. A record whose every entity is rejected exports
nothing and is still marked processed, as for a response with no
entities.
async
¶
Extract entities from text with a single LLM call.
The entity list is read from result_key, or from the only value when
the response has exactly one key; any other shape yields no entities.
null reads as no entities and a lone object as a one-entity list.
HTML stripping and truncation run in a worker thread, since both are
CPU-bound and token counting may load encoding data on first use.
Raises:
| Type | Description |
|---|---|
LLMError
|
The completion failed, or the entity list is not a list of objects (for example a string or a list of strings). The error propagates instead of reading as "no entities", so the pipeline leaves the record unmarked and retries it on the next run rather than recording it as processed with nothing exported. |
InMemoryLLMResponseCache(max_entries: int | None = None)
Bases: AsyncLLMResponseCache
A cache that lives as long as the process, optionally bounded.
With max_entries, the least recently used response is evicted once the
cache is full. Responses are copied in and out, so a caller that changes a
returned dict cannot change what is cached.
Create an empty cache.
Raises:
| Type | Description |
|---|---|
ValueError
|
|
Bases: AsyncLLMResponseCache
A cache kept in a SQLite file, so responses survive between runs.
The file and its parent folder are created on first use, and responses are
stored as JSON. Every SQLite failure, including a file that is not a
database, raises :class:~sci_etl_core.exceptions.LLMCacheError. Close it
with :meth:aclose, for example by listing it in the pipeline's
closeables. Use each instance from one event loop.
dataclass
¶
CachingLLMClient(
client: AsyncLLMClient,
cache: AsyncLLMResponseCache,
model: str | None = None,
logger: Callable[[str], None] | None = None,
)
Bases: AsyncLLMClient
Serve repeated completions from a cache instead of calling the LLM again.
A request is keyed on model and both prompts (:func:response_cache_key).
The timeout is not part of the key. Only successful responses are cached,
so a failed call is retried the next time it is made. Change model when
anything else that shapes the response changes, such as the temperature,
for example "gpt-4o-mini@t0.2".
The cache never fails a completion: an exception from the cache is logged
as LLM cache <get|set> failed: <error>, counted in :attr:stats, and the
call goes to the LLM as on a miss. :attr:usage is the wrapped client's, so
cache hits cost no tokens.
Wrap client.
model defaults to the wrapped client's model attribute.
Raises:
| Type | Description |
|---|---|
ValueError
|
|