Writers API¶

The writers.py module is Foo’s output serialization layer. It contains writer classes that take processed content and write it to output files, with Markdown as the primary documented output format.

This module should remain focused on output writing. It should not contain Streamlit layout code, database persistence behavior, web retrieval logic, HTML scraping logic, document loading logic, or AI-provider request logic.

🧭 Purpose¶

The Writers API provides a clean boundary for exporting content from Foo workflows.

The module supports:

Preparing output paths.
Validating required write inputs.
Writing text-oriented content.
Writing Markdown content.
Returning output paths or documented write results.
Keeping serialization behavior separate from loaders, fetchers, scrapers, generators, and data-management classes.
Logging handled write failures through Foo’s structured error pattern.

Writers should be used when a workflow has already produced content and needs to save it outside the active Python or Streamlit session.

🧱 Module Role¶

writers.py sits at the end of many Foo workflows.

Load / fetch / scrape / generate / retrieve
                  |
                  v
          processed content
                  |
                  v
             writers.py
                  |
                  v
          output file artifacts

A writer should not decide how to fetch, load, scrape, generate, or store data. It should receive content and write it in a predictable format.

🧩 Class Overview¶

The module defines two primary classes:

Class	Purpose
`Writer`	Base writer class for output-file behavior.
`MarkdownWriter`	Concrete writer class for Markdown output.

Use Writer as the common conceptual base. Use MarkdownWriter when the workflow needs Markdown serialization.

⚙️ Writer¶

Writer is the base class for output writing.

It represents the common state and behavior shared by output writers. A writer typically tracks:

Output path.
Output filename.
Output content.
File extension.
Encoding.
Write result state.
Error state where applicable.

The base writer is useful when future output formats need to share common behavior with Markdown writing.

Conceptual usage:

from writers import Writer

writer = Writer()

print(writer)

In normal application workflows, use a concrete writer such as MarkdownWriter.

📝 MarkdownWriter¶

MarkdownWriter writes content to Markdown files.

Use it when Foo needs to export:

Scraped page summaries.
Link inventories.
Extracted tables.
Loaded document summaries.
Generated AI output.
Research notes.
Documentation drafts.
API result summaries.
Processing reports.
Workflow artifacts.

Conceptual usage:

from writers import MarkdownWriter

writer = MarkdownWriter()

writer.write(
    path="outputs/example.md",
    content="# Example Output\n\nThis file was generated by Foo.\n",
)

Use the generated API reference below for the exact method signature implemented in the source.

📄 Output Content Sources¶

Writer classes usually consume content produced by other Foo modules.

Source Module	Content That May Be Written
`loaders.py`	Loaded document text, metadata summaries, document inventories.
`fetchers.py`	API results, web page summaries, link inventories, public-data snapshots.
`scrapers.py`	Extracted headings, paragraphs, tables, links, images, and page text.
`generators.py`	Generated responses, summaries, translations, transcriptions, documentation drafts.
`data.py`	Table exports, stored metadata, retrieval results, workflow history.
`models.py`	Serialized schema objects, tool definitions, message records, prompt records.

The writer layer should not re-create the behavior of these modules. It should only serialize their results.

🧾 Markdown Output Guidance¶

Markdown output should be readable, portable, and easy to inspect in a repository.

Recommended practices:

Start with one # page title.
Use ## headings for major sections.
Use tables for structured records.
Use fenced code blocks for logs, JSON, SQL, or command output.
Keep generated Markdown conservative and GitHub-compatible.
Avoid raw HTML unless the output requires it.
Escape or fence content that may be interpreted as Markdown syntax.
Keep file names stable and descriptive.
Use relative paths for internal documentation references.
Keep generated content deterministic where possible.

Good Markdown output is useful in both MkDocs and GitHub.

🧪 Example: Write a Scrape Summary¶

from writers import MarkdownWriter

writer = MarkdownWriter()

content = """# Scrape Summary

## Source

https://example.com

## Extracted Headings

- Overview
- Installation
- Usage
- API Reference
"""

writer.write(
    path="outputs/scrape-summary.md",
    content=content,
)

This pattern is useful after scraping page headings, paragraphs, links, or tables.

🧪 Example: Write a Fetch Result Summary¶

from writers import MarkdownWriter

writer = MarkdownWriter()

content = """# Fetch Result

## Metadata

| Field | Value |
| --- | --- |
| URL | https://example.com |
| Status Code | 200 |
| Encoding | utf-8 |

## Notes

The page was fetched successfully.
"""

writer.write(
    path="outputs/fetch-result.md",
    content=content,
)

This pattern is useful when a web or public-data result needs to be saved as a reviewable artifact.

🧪 Example: Write Generated Text¶

from writers import MarkdownWriter

writer = MarkdownWriter()

generated_text = "Foo separates loading, fetching, scraping, generation, persistence, and writing."

content = f"""# Generated Summary

{generated_text}
"""

writer.write(
    path="outputs/generated-summary.md",
    content=content,
)

This pattern is useful after a generator workflow produces a summary, explanation, translation, or draft.

🧪 Example: Write a Table Export¶

from writers import MarkdownWriter

writer = MarkdownWriter()

rows = [
    ("loaders.py", "Document ingestion"),
    ("fetchers.py", "External retrieval"),
    ("scrapers.py", "HTML extraction"),
    ("writers.py", "Output serialization"),
]

lines = [
    "# Module Summary",
    "",
    "| Module | Purpose |",
    "| --- | --- |",
]

for module, purpose in rows:
    lines.append(f"| `{module}` | {purpose} |")

writer.write(
    path="outputs/module-summary.md",
    content="\n".join(lines),
)

This pattern is useful for exporting table-oriented data from SQLite, API results, or inspection workflows.

🔄 Writer Workflow¶

A typical writer workflow follows this sequence:

Processed result
      |
      v
Build output content
      |
      v
Choose output path
      |
      v
Validate content and path
      |
      v
Write file
      |
      v
Return or display output status

Example with module boundaries:

fetchers.WebFetcher
      |
      v
Result or extracted text
      |
      v
Markdown content builder
      |
      v
writers.MarkdownWriter
      |
      v
outputs/page-summary.md

The content builder may live in the UI, a workflow helper, or a dedicated formatting function. The writer should perform the file write.

🧩 Relationship to Loaders¶

Loaders produce document content that can be exported.

Examples:

Export plain text loaded from a file.
Export PDF page summaries.
Export CSV row summaries.
Export Markdown copies of Word or PowerPoint content.
Export metadata inventories for loaded documents.

The loader should return content. The writer should save it.

🧩 Relationship to Fetchers¶

Fetchers retrieve external data that can be exported.

Examples:

Export a web page fetch summary.
Export a list of discovered links.
Export API result summaries.
Export public-data records.
Export weather or geospatial query results.
Export astronomy or environmental data snapshots.

The fetcher should retrieve and normalize the data. The writer should serialize the result.

🧩 Relationship to Scrapers¶

Scrapers extract HTML elements that can be exported.

Examples:

Export headings as a page outline.
Export paragraphs as readable text.
Export tables as Markdown tables.
Export links as an inventory.
Export images as an asset list.
Export blockquotes as source excerpts.

The scraper should return extracted values. The writer should format and write them.

🧩 Relationship to Generators¶

Generators produce AI-assisted output that can be exported.

Examples:

Summaries.
Explanations.
Translations.
Transcriptions.
Documentation drafts.
Search-assisted answers.
Research notes.
Prompt outputs.

The generator should return generated content. The writer should save the content in a file format.

🧩 Relationship to Persistence¶

Persistence workflows can provide stored records for export.

Examples:

Export SQLite table rows.
Export source inventories.
Export processing history.
Export retrieval results.
Export semantic-search results.
Export generation records.

The data layer should retrieve records. The writer should format and write them.

🔐 Error Handling¶

Writer classes should use Foo’s structured error logging pattern where handled exception paths exist.

Preferred pattern:

except Exception as e:
    exception = Error(e)
    exception.module = "writers"
    exception.cause = "MarkdownWriter"
    exception.method = "write( self, path: str, content: str ) -> str | None"
    Logger().write(exception)
    raise exception

Error metadata should be stable and reviewer-safe.

Do not include:

Full file contents.
Raw document bodies.
Full generated responses when they may contain private content.
API keys.
Tokens.
Credentials.
Sensitive user-provided values.
Large binary content.

The logging purpose is to identify the failing module, class, and method without capturing sensitive output material.

🛡️ Safe Writing Guidance¶

When adding or modifying writer methods:

Validate the output path.
Validate the content argument.
Create parent directories only when intended.
Use explicit encodings, preferably UTF-8.
Avoid overwriting files unless that behavior is expected.
Return a documented path or status when useful.
Do not log full output contents.
Keep serialization behavior deterministic.
Keep UI rendering out of writers.py.
Keep database retrieval in data.py.
Keep provider calls in generators.py.
Keep web retrieval in fetchers.py.
Keep document loading in loaders.py.

🧪 Testing Writers¶

Writer tests should verify normal and failure behavior.

Test cases should include:

Writing a valid Markdown file.
Writing to a nested output directory.
Writing an empty string when allowed.
Rejecting missing path values.
Rejecting missing content values when required.
Handling invalid paths.
Handling permission errors where feasible.
Preserving UTF-8 content.
Returning the documented result.
Logging handled exceptions before re-raising where the pattern is used.

For Markdown output, inspect the generated file to confirm:

Headings render correctly.
Tables render correctly.
Code fences are closed.
Relative links remain valid.
Special characters are preserved.
No unintended raw HTML is introduced.

🧪 Return Shape Guidance¶

Writer methods should return predictable values.

Good return shapes include:

str path to the written file.
Path object for the written file.
bool status.
None for write procedures that intentionally do not return a meaningful value.

If a writer returns a path, document it clearly. If a writer only mutates the filesystem, annotate it as None and avoid adding a Returns: docstring section unless the return value is meaningful.

📦 Dependency Guidance¶

The writer layer should remain dependency-light.

Appropriate dependencies include:

pathlib
Standard file I/O helpers.
Encoding utilities.
Optional Markdown formatting helpers if needed.

Avoid adding dependencies for:

Streamlit rendering.
HTTP retrieval.
HTML scraping.
AI provider clients.
Database connections.
Vector stores.
Cloud storage clients unless the writer is explicitly a cloud writer.

If a future writer supports additional output targets, keep each target-specific writer isolated behind its own class.

🧰 Adding a New Writer¶

When adding a new writer class:

Add the class to writers.py.
Inherit from Writer when common writer behavior applies.
Initialize default state in __init__.
Use a clear method name such as write(...).
Validate required arguments.
Use explicit output encoding where text is written.
Return a documented output path or status if meaningful.
Preserve existing exception behavior.
Add structured logging to handled exception paths.
Avoid logging full output content.
Update user-facing documentation if the writer is exposed.
Confirm the generated API page renders without Griffe warnings.

📖 API Documentation¶

The generated API reference for this module is rendered below.