Excel Data
excel
¶
Assembly: Mappy
Filename: excel.py
Author: Terry D. Eppler
Created: 05-31-2022
Last Modified By: Terry D. Eppler
Last Modified On: 05-01-2025
Mappy is a python framework encapsulating the Google Maps functionality.
Copyright © 2022 Terry Eppler
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
You can contact me at: terryeppler@gmail.com or eppler.terry@epa.gov
Classes¶
Excel
¶
Excel(api: str, cache: Optional[BaseCache] = None)
Read, write, and enrich spreadsheet location records.
Purpose
Provides file-based and dataframe-oriented helpers for reading CSV/XLSX data, preserving string fields, resolving addresses through Geocoder and Places fallback services, appending canonical geospatial output columns, writing enriched files, and storing summary metrics for Streamlit display or downstream reporting.
Attributes:
| Name | Type | Description |
|---|---|---|
api_key |
str
|
Google Maps Platform API key supplied to the helper. |
maps |
Optional[Maps]
|
Maps gateway used by Geocoder and Places wrappers. |
geocoder |
Optional[Geocoder]
|
Geocoder service used as the primary location resolver. |
places |
Optional[Place]
|
Places service used as the fallback location resolver. |
input_path |
Optional[str]
|
Input CSV/XLSX path used by the latest enrichment operation. |
output_path |
Optional[str]
|
Output CSV/XLSX path used by the latest enrichment operation. |
dataframe |
Optional[DataFrame]
|
DataFrame read or enriched by the latest operation. |
worksheet |
Optional[str]
|
Excel worksheet name or index context used by read/write operations. |
output |
Optional[Dict[str, List]]
|
Output-column container from the latest enrichment operation. |
address |
Optional[str]
|
Address column name used by address-based enrichment. |
cache |
Optional[BaseCache]
|
Optional cache backend shared by Geocoder and Places. |
file_path |
Optional[str]
|
File path used by the latest read or write operation. |
last_summary |
Optional[Dict]
|
Summary metrics from the latest enrichment result. |
city |
Optional[str]
|
City column name or current city value used by enrichment. |
state |
Optional[str]
|
State column name or current state value used by enrichment. |
country |
Optional[str]
|
Country column name or current country value used by enrichment. |
Initialize the Excel enrichment helper.
Purpose
Stores the supplied API key, optional cache backend, shared Maps gateway, Geocoder wrapper, Places fallback wrapper, and runtime state used by later read, write, resolution, enrichment, and summary methods.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
api
|
str
|
Google Maps Platform API key. |
required |
cache
|
Optional[BaseCache]
|
Optional cache backend shared by Geocoder and Places. |
None
|
Source code in excel.py
Methods:¶
read
¶
Read a CSV or Excel file into a cleaned DataFrame.
Purpose
Loads CSV or XLSX input while preserving string-like values, including leading zeros. Object columns are filled, converted to strings, and stripped so downstream geocoding and column-detection workflows operate on stable text values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
Input file path ending in |
required |
sheet
|
Optional[str]
|
Optional worksheet name for Excel input. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
pd.DataFrame: Loaded and cleaned DataFrame. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation, file reading, or dataframe normalization fails. |
Source code in excel.py
write
¶
Write a DataFrame to CSV or Excel.
Purpose
Copies the supplied DataFrame, stores the output path and worksheet context
on the instance, and writes the data with the index suppressed. CSV output
uses to_csv and non-CSV output uses to_excel with a default sheet
name when one is not supplied.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
DataFrame to persist. |
required |
path
|
str
|
Output file path ending in |
required |
sheet
|
Optional[str]
|
Optional Excel worksheet name. |
required |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or file writing fails. |
Source code in excel.py
create_outputs
¶
Create the standard geospatial output-column container.
Purpose
Builds the dictionary of empty lists used during file enrichment. Each list represents one canonical output column appended to the source DataFrame, including formatted address, coordinates, place metadata, administrative components, row-level status, and row-level error fields.
Returns:
| Type | Description |
|---|---|
Dict[str, List]
|
Dict[str, List]: Output-column container with empty value lists. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when output-container construction fails. |
Source code in excel.py
append_empty
¶
append_empty(
outputs: Dict[str, List],
status: str = "skipped_empty",
error: Optional[str] = None,
) -> None
Append an empty geocode result to the output container.
Purpose
Adds placeholder values to every canonical geospatial output column for a source row that cannot or should not be geocoded. The method preserves row alignment between the original DataFrame and appended enrichment columns.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
outputs
|
Dict[str, List]
|
Output-column container created by |
required |
status
|
str
|
Row-level geocode status to append. |
'skipped_empty'
|
error
|
Optional[str]
|
Optional row-level error message. |
None
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or output mutation fails. |
Source code in excel.py
append_result
¶
append_result(
outputs: Dict[str, List],
result: Dict,
status: str,
error: Optional[str] = None,
) -> None
Append one resolved geospatial result to the output container.
Purpose
Extracts canonical geocode or Places result fields and appends them to the enrichment output lists. The method keeps each output column aligned with the source DataFrame row and appends row-level status and error metadata alongside the resolved location values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
outputs
|
Dict[str, List]
|
Output-column container created by |
required |
result
|
Dict
|
Flattened Geocoder or Places result dictionary. |
required |
status
|
str
|
Row-level resolution status. |
required |
error
|
Optional[str]
|
Optional row-level error message. |
None
|
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or output mutation fails. |
Source code in excel.py
summarize
¶
Build summary metrics for an enriched DataFrame.
Purpose
Counts total rows and row-level geocoding statuses from the latest enrichment result. The summary is stored on the instance for Streamlit display, reporting, audit checks, and downstream validation of enrichment coverage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Enriched DataFrame containing optional |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Dict |
Dict
|
Summary metrics for the enrichment result. |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or summary generation fails. |
Source code in excel.py
detect_columns
¶
Suggest likely location columns from a DataFrame.
Purpose
Inspects DataFrame column names using normalized lowercase comparisons and returns likely address, city, state, and country column names. The result supports Streamlit defaults, manual enrichment setup, and file-ingestion workflows where users upload heterogeneous location datasets.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
DataFrame to inspect. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Dict |
Dict
|
Suggested column names keyed by |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation or column inspection fails. |
Source code in excel.py
resolve_address
¶
Resolve one free-form address with Geocoder and Places fallback.
Purpose
Attempts to resolve a free-form address through Geocoder first, then falls back to Places Text Search when geocoding fails. The method returns a standard row-resolution dictionary containing a result payload, status, and error text so file enrichment can continue without failing an entire batch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
address
|
str
|
Free-form address text. |
required |
country
|
Optional[str]
|
Optional ISO-2 country bias. Defaults to |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
Dict |
Dict
|
Row-level resolution result with |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when top-level validation or resolution setup fails. |
Source code in excel.py
resolve_city_state_country
¶
Resolve one structured city/state/country location.
Purpose
Attempts to resolve a structured location through Geocoder first, then falls back to Places Text Search using a composed query when geocoding fails. The method returns a standard row-resolution dictionary so file enrichment can preserve row-level status without stopping the batch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
city
|
str
|
City or locality value. |
required |
state
|
Optional[str]
|
Optional state, province, region, or territory value. |
required |
country
|
str
|
Country name or ISO-2 country code. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Dict |
Dict
|
Row-level resolution result with |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when top-level resolution setup fails. |
Source code in excel.py
enrich_from_address
¶
enrich_from_address(
inpath: str,
outpath: str,
address: str,
sheet: Optional[str],
cntry: Optional[str],
) -> DataFrame
Enrich a file using one free-form address column.
Purpose
Reads a CSV/XLSX file, validates the address column, resolves each populated address through Geocoder and Places fallback, appends canonical geospatial output columns, stores enrichment state and summary metrics, and writes the enriched DataFrame to the requested output path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inpath
|
str
|
Input CSV/XLSX path containing the address column. |
required |
outpath
|
str
|
Output CSV/XLSX path to write enriched results. |
required |
address
|
str
|
Column name containing free-form address text. |
required |
sheet
|
Optional[str]
|
Optional Excel worksheet name. |
required |
cntry
|
Optional[str]
|
Optional column name containing country-bias values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
pd.DataFrame: Enriched DataFrame written to |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation, reading, enrichment, summary, or writing fails. |
Source code in excel.py
637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 | |
enrich
¶
enrich(
inpath: str,
outpath: str,
city: str,
state: Optional[str],
cntry: str,
sheet: Optional[str] = None,
) -> DataFrame
Enrich a file using city, state, and country columns.
Purpose
Reads a CSV/XLSX file, validates the configured city, optional state, and country columns, resolves each populated location through Geocoder and Places fallback, appends canonical geospatial output columns, stores enrichment state and summary metrics, and writes the enriched DataFrame to the requested output path.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
inpath
|
str
|
Input CSV/XLSX path containing location columns. |
required |
outpath
|
str
|
Output CSV/XLSX path to write enriched results. |
required |
city
|
str
|
Column name containing city or locality values. |
required |
state
|
Optional[str]
|
Optional column name containing state, province, region, or territory values. |
required |
cntry
|
str
|
Column name containing country names or ISO-2 country codes. |
required |
sheet
|
Optional[str]
|
Optional Excel worksheet name. |
None
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
pd.DataFrame: Enriched DataFrame written to |
Raises:
| Type | Description |
|---|---|
Error
|
Raised after logging when validation, reading, enrichment, summary, or writing fails. |
Source code in excel.py
722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 | |
Functions:¶
throw_if
¶
Validate that a required Excel workflow value is present.
Purpose
Provides a lightweight guard for required file paths, dataframes, output containers, column names, and geocoding inputs before spreadsheet enrichment operations continue. The function rejects missing objects and blank strings so file and dataframe workflows fail with clear validation errors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the argument being validated. |
required |
value
|
object
|
Runtime value to validate. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
Raised when |