Caching
caches
¶
Assembly: Mappy
Filename: caches.py
Author: Terry D. Eppler
Created: 05-31-2022
Last Modified By: Terry D. Eppler
Last Modified On: 05-01-2025
Mappy is a python framework encapsulating the Google Maps functionality.
Copyright © 2022 Terry Eppler
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
You can contact me at: terryeppler@gmail.com or eppler.terry@epa.gov
Classes¶
BaseCache
¶
Define the common cache interface used by Mappy services.
Purpose
Provides the shared contract for cache backends that map string keys to JSON-serializable dictionary values. Geocoding, Places, weather, fetcher, and spreadsheet-enrichment workflows can use this interface without depending on a specific in-memory or SQLite implementation.
Notes
Subclasses implement storage-specific behavior for get, set,
delete, clear, and stats.
Methods:¶
get
¶
Return a cached value by key.
Purpose
Defines the read contract implemented by cache backends. Concrete implementations should return cached dictionary payloads only when the key exists and the record has not expired.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key to read. |
required |
Returns:
| Type | Description |
|---|---|
Optional[Dict[str, Any]]
|
Optional[Dict[str, Any]]: Cached dictionary payload, or |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
Raised when the base interface is called directly. |
Source code in caches.py
set
¶
Persist a value by cache key.
Purpose
Defines the write contract implemented by cache backends. Concrete implementations should store JSON-serializable dictionary payloads and apply the optional time-to-live value when provided.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key to write. |
required |
value
|
Dict[str, Any]
|
JSON-serializable dictionary payload. |
required |
ttl
|
Optional[int]
|
Optional time-to-live in seconds. |
None
|
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
Raised when the base interface is called directly. |
Source code in caches.py
delete
¶
Delete a cache record by key.
Purpose
Defines the deletion contract implemented by cache backends. Concrete implementations should remove the matching record without affecting unrelated keys or namespaces.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key to delete. |
required |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
Raised when the base interface is called directly. |
Source code in caches.py
clear
¶
Clear cache records.
Purpose
Defines the cache-clearing contract implemented by cache backends. Concrete implementations should remove all records when no namespace is supplied, or only namespace-prefixed records when a namespace is supplied.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
Optional[str]
|
Optional namespace prefix to clear. |
None
|
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
Raised when the base interface is called directly. |
Source code in caches.py
contains
¶
Determine whether a key resolves to a usable cached value.
Purpose
Provides a common implementation that validates the key and delegates to
get. Backends can override this method for optimized lookups, but the
default behavior preserves TTL-aware semantics by relying on get.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key to test. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when key validation or backend lookup fails. |
Source code in caches.py
namespace_key
¶
Create a namespace-qualified cache key.
Purpose
Builds a stable cache key that separates values by workflow or service area. Geocoding, places, weather, distance, and fetcher workflows can use namespaces to avoid collisions when raw keys overlap.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
str
|
Logical cache namespace, such as |
required |
key
|
str
|
Raw cache key. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Namespace-qualified cache key. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or key construction fails. |
Source code in caches.py
stats
¶
Return cache diagnostics.
Purpose
Defines the diagnostics contract implemented by cache backends. Concrete implementations should return backend name, entry counts, expiration counts, and available hit or mutation counters.
Returns:
| Type | Description |
|---|---|
Dict[str, Any]
|
Dict[str, Any]: Cache diagnostics. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
Raised when the base interface is called directly. |
Source code in caches.py
InMemoryCache
¶
Bases: BaseCache
Provide a process-local dictionary cache.
Purpose
Implements the Mappy cache contract using an in-memory dictionary with optional TTL handling and runtime diagnostics. This backend supports fast lookups during a single Streamlit or Python process and is useful for geocoding, Places, fetcher, and enrichment workflows that do not require durable cache persistence.
Attributes:
| Name | Type | Description |
|---|---|---|
_store |
Dict[str, Dict[str, Any]]
|
Dictionary containing cache metadata records by key. |
_hits |
int
|
Number of successful cache reads. |
_misses |
int
|
Number of cache misses or expired reads. |
_sets |
int
|
Number of cache writes. |
_deletes |
int
|
Number of cache deletions. |
Initialize the in-memory cache backend.
Purpose
Creates the backing dictionary and resets runtime diagnostics used by
get, set, delete, clear, and stats. The cache starts
empty and remains available for the lifetime of the current Python
process.
Source code in caches.py
Methods:¶
now
¶
Return the current Unix timestamp.
Purpose
Provides a single timestamp source for cache record creation, TTL expiration checks, last-access tracking, and diagnostics.
Returns:
| Name | Type | Description |
|---|---|---|
float |
float
|
Current Unix timestamp. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when timestamp retrieval fails. |
Source code in caches.py
make_record
¶
Create a cache metadata record.
Purpose
Packages a cache key, JSON-serializable payload, timestamps, optional
expiration time, hit counter, and last-access marker into the internal
record shape used by InMemoryCache.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
value
|
Dict[str, Any]
|
JSON-serializable dictionary payload. |
required |
ttl
|
Optional[int]
|
Optional time-to-live in seconds. |
None
|
Returns:
| Type | Description |
|---|---|
Dict[str, Any]
|
Dict[str, Any]: Cache metadata record. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or record construction fails. |
Source code in caches.py
is_expired
¶
Determine whether an in-memory record has expired.
Purpose
Reads the record's expires_at metadata and compares it with the
current timestamp. The method keeps TTL handling centralized for reads,
statistics, and cleanup behavior.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
record
|
Dict[str, Any]
|
Cache metadata record. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or timestamp comparison fails. |
Source code in caches.py
get
¶
Return a cached value by key.
Purpose
Looks up an in-memory cache record, removes expired records, updates hit and last-access metadata for successful reads, and returns only the stored payload to callers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
Returns:
| Type | Description |
|---|---|
Optional[Dict[str, Any]]
|
Optional[Dict[str, Any]]: Cached payload, or |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or lookup fails. |
Source code in caches.py
set
¶
Set a cache value by key.
Purpose
Stores a JSON-serializable dictionary payload using the in-memory record shape and optional TTL metadata. The method increments set diagnostics after a successful write.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
value
|
Dict[str, Any]
|
JSON-serializable dictionary payload. |
required |
ttl
|
Optional[int]
|
Optional time-to-live in seconds. |
None
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or write handling fails. |
Source code in caches.py
delete
¶
Delete a cache entry by key.
Purpose
Removes an in-memory cache record when the key exists and updates the delete counter. Missing keys are ignored so cleanup paths can call this method safely.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or deletion fails. |
Source code in caches.py
clear
¶
Clear all cache entries or a namespace subset.
Purpose
Deletes every in-memory record when no namespace is supplied, or deletes only keys beginning with the namespace prefix when a namespace is supplied. This supports targeted cache resets for service-specific workflows.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
Optional[str]
|
Optional namespace prefix. |
None
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when clear handling fails. |
Source code in caches.py
contains
¶
Determine whether a key exists and is not expired.
Purpose
Delegates to get so the containment check honors TTL expiration,
deletes expired records, and returns the same result a read operation
would observe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or lookup fails. |
Source code in caches.py
stats
¶
Return in-memory cache diagnostics.
Purpose
Reports backend type, active entry count, expired entry count, hit count, miss count, write count, and deletion count for UI display, debugging, and operational inspection.
Returns:
| Type | Description |
|---|---|
Dict[str, Any]
|
Dict[str, Any]: Cache diagnostics. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when diagnostics generation fails. |
Source code in caches.py
namespace_key
¶
Create a namespace-qualified cache key.
Purpose
Builds a stable cache key that separates values by workflow or service area. Geocoding, places, weather, distance, and fetcher workflows can use namespaces to avoid collisions when raw keys overlap.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
str
|
Logical cache namespace, such as |
required |
key
|
str
|
Raw cache key. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Namespace-qualified cache key. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or key construction fails. |
Source code in caches.py
SQLiteCache
¶
Bases: BaseCache
Persist cache records in a SQLite database.
Purpose
Implements the Mappy cache contract using a local SQLite key-value table. Values are stored as JSON strings with creation, update, expiration, hit-count, and last-access metadata so cache state can survive application restarts and support diagnostics.
Attributes:
| Name | Type | Description |
|---|---|---|
path |
Optional[str]
|
SQLite database file path. |
_conn |
Connection
|
Active SQLite connection used by cache operations. |
Initialize the SQLite cache backend.
Purpose
Stores the database path, creates the containing directory when needed, opens the SQLite connection, and initializes or migrates the cache table used by durable cache operations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
SQLite database file path. |
'mappy_cache.sqlite'
|
Source code in caches.py
Methods:¶
initialize
¶
Create the SQLite cache table when needed.
Purpose
Ensures the durable key-value table exists with payload, timestamp, expiration, hit-count, and last-access columns. The method also invokes migration logic so older cache tables remain compatible with current diagnostics and TTL behavior.
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when table creation, migration, or commit handling fails. |
Source code in caches.py
migrate
¶
Add missing metadata columns to older SQLite cache tables.
Purpose
Reads the current kv schema and adds metadata columns that may be
missing from legacy tables. This keeps existing cache databases usable
after upgrades without requiring users to delete local cache files.
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when schema inspection or migration fails. |
Source code in caches.py
now
¶
Return the current Unix timestamp.
Purpose
Provides a single timestamp source for SQLite cache creation, expiration, last-access updates, purge operations, and diagnostics.
Returns:
| Name | Type | Description |
|---|---|---|
float |
float
|
Current Unix timestamp. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when timestamp retrieval fails. |
Source code in caches.py
get
¶
Return a cached SQLite value by key.
Purpose
Reads a JSON payload from the SQLite cache, deletes expired rows, updates hit-count and last-access metadata for successful reads, and returns the decoded dictionary payload.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
Returns:
| Type | Description |
|---|---|
Optional[Dict[str, Any]]
|
Optional[Dict[str, Any]]: Cached payload, or |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation, SQL execution, or JSON decoding fails. |
Source code in caches.py
set
¶
Set a SQLite cache value by key.
Purpose
Serializes a dictionary payload to JSON and upserts it into the SQLite cache table with creation, update, and optional expiration metadata. The method preserves durable cache behavior across application sessions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
value
|
Dict[str, Any]
|
JSON-serializable dictionary payload. |
required |
ttl
|
Optional[int]
|
Optional time-to-live in seconds. |
None
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation, JSON serialization, SQL execution, or commit handling fails. |
Source code in caches.py
delete
¶
Delete a SQLite cache entry by key.
Purpose
Removes a durable cache record from the SQLite key-value table and commits the change. Missing keys are ignored by SQLite so cleanup paths can safely call this method.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation, SQL execution, or commit handling fails. |
Source code in caches.py
clear
¶
Clear all SQLite cache entries or a namespace subset.
Purpose
Deletes every durable cache row when no namespace is supplied, or deletes only rows with keys matching the namespace prefix when a namespace is supplied. This supports targeted cleanup for service-specific workflows.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
Optional[str]
|
Optional namespace prefix. |
None
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when SQL execution or commit handling fails. |
Source code in caches.py
contains
¶
Determine whether a SQLite cache key exists and is not expired.
Purpose
Validates the supplied key and delegates to get so containment checks
honor TTL expiration and produce the same result a read operation would
observe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
Cache key. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or lookup fails. |
Source code in caches.py
purge_expired
¶
Delete expired SQLite cache entries.
Purpose
Counts and removes rows whose expiration timestamp has passed. The method supports manual cache maintenance and diagnostics for durable cache backends.
Returns:
| Name | Type | Description |
|---|---|---|
int |
int
|
Number of expired rows deleted. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when SQL execution or commit handling fails. |
Source code in caches.py
stats
¶
Return SQLite cache diagnostics.
Purpose
Reports backend type, database path, total row count, expired row count, and accumulated hit count for UI display, debugging, and operational inspection.
Returns:
| Type | Description |
|---|---|
Dict[str, Any]
|
Dict[str, Any]: SQLite cache diagnostics. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when diagnostics queries fail. |
Source code in caches.py
close
¶
Close the SQLite cache connection.
Purpose
Closes the active SQLite connection used by the durable cache backend. This supports explicit cleanup in scripts, tests, and long-running application workflows that need deterministic connection shutdown.
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when connection close handling fails. |
Source code in caches.py
namespace_key
¶
Create a namespace-qualified cache key.
Purpose
Builds a stable cache key that separates values by workflow or service area. Geocoding, places, weather, distance, and fetcher workflows can use namespaces to avoid collisions when raw keys overlap.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
str
|
Logical cache namespace, such as |
required |
key
|
str
|
Raw cache key. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Namespace-qualified cache key. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or key construction fails. |
Source code in caches.py
Functions:¶
throw_if
¶
Validate that a required cache value is present.
Purpose
Provides a lightweight guard used by cache backends before keys, namespaces, records, and payloads are read or persisted. The function raises clear validation errors for missing objects and blank strings so cache callers fail before corrupt or ambiguous keys are created.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the argument being validated. |
required |
value
|
object
|
Runtime value to validate. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
Raised when |