Graph API¶
This page documents the pipeline-facing graph contracts.
The important boundary is the protocol and DTO layer, not the Neo4j backend.
GraphReader.get_neighborhood() returns a SubGraph, which is the canonical transport object for structural context
passed into Phase 3, Phase 4, and fusion logic.
Key Semantics¶
SubGraphis a serializable neighborhood snapshot, not a live graph handle.center_ididentifies the query center; the center node does not need to be duplicated innodes.nodesandtriplesexpose the retrieved local context in pipeline-native types.- Storage backends may vary, but callers should rely only on the documented
SubGraphandGraphReadercontracts.
Envelope Construction and Traversal¶
When GraphReader.get_neighborhood() constructs a SubGraph envelope (e.g., for Phase 3 and Phase 4 LLM context), it
explicitly filters out structural nodes (Chunk, Chapter, Document).
Why filter structural nodes?
- Structural nodes like
Chunkcarry heavy metadata payloads (e.g., 4096-dimensional dense embeddings,phase2_processedflags, source IDs). - If traversed, these nodes would leak into the LLM prompt via the envelope formatting (
format_envelope), causing severe context window overflows and diluting the semantic reasoning task. - By ignoring structural nodes during the
apoc.path.subgraphAlltraversal, the resultingSubGraphstrictly contains semantic entity-to-entity and component-to-component relationships, which is the exact context needed for LLM relation extraction and argument mining.
Batch Persistence and Bulk I/O¶
The write interface GraphWriter exposes explicit plural persistence contracts (upsert_chunks, upsert_entities, upsert_triples, upsert_relations, upsert_argument_components, upsert_communities) alongside singular operations.
Key write invariants:
- Single-Transaction Bulk Writes: Plural write operations execute within a single Cypher transaction using
UNWIND $batch AS .... This reduces transaction management and round-trip network overhead by up to \(95\%\) during large ingestions. - DRY Singular Delegation: To avoid query duplication and ensure consistent label/property mapping, singular methods (
upsert_entity,upsert_triple,upsert_chunk, etc.) wrap inputs in single-element lists and delegate to the plural batch implementations. - Explicit Contracts over Global Flags: There is no global
enable_batchingflag. Phases decide whether to batch data (typically via Python's standarditertools.batched) based on their algorithmic constraints.
For the architectural rationale and benchmarks, see ADR 0017: Explicit Batch Contracts & Bulk I/O.
Contracts¶
episteme_pipeline.contracts.domain.SubGraph
¶
Bases: BaseModel
Depth-limited neighborhood returned by graph read operations.
This model is the pipeline-level transport object for structural context. It intentionally does not expose backend-specific graph handles or third- party graph objects. Consumers should treat it as a compact, serializable snapshot of a local property-graph neighborhood.
Parameters¶
center_id
Identifier of the node around which the neighborhood was requested.
The center node itself is referenced here and is not required to appear
in nodes.
nodes
Neighbor nodes materialized for the neighborhood. Implementations
should return the reachable context nodes needed by downstream phases.
Ordering is not semantically significant.
triples
Directed relations contained in the neighborhood snapshot. These are
graph edges represented in the pipeline's canonical L2Triple shape
so downstream components can inspect structure without depending on the
persistence backend.
depth
Maximum traversal depth that was requested to construct the
neighborhood.
Source code in packages/episteme-pipeline/episteme_pipeline/contracts/domain.py
212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 | |
episteme_pipeline.protocols.graph_store.GraphReader
¶
Bases: GraphHandle, ABC
Read surface for graph-backed pipeline components.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | |
get_chunk_entities(chunk_id)
abstractmethod
async
¶
Return all entities extracted from a specific chunk.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
106 107 108 109 | |
get_entity_envelopes(entity_id)
abstractmethod
async
¶
Return the stored textual envelopes for entity_id.
An envelope is the sentence-level context a mention was extracted from.
Phase 4 maturation needs these to synthesise an entity description, and
used to guard the call with hasattr because the contract did not
declare it. Implementations with no envelope storage return an
empty list.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
94 95 96 97 98 99 100 101 102 103 104 | |
get_neighborhood(node_id, depth=1)
abstractmethod
async
¶
Return a depth-limited structural neighborhood around a node.
Parameters¶
node_id
Identifier of the center node whose local graph context should be
retrieved.
depth
Maximum hop distance from node_id to include in the returned
neighborhood.
Returns¶
SubGraph
Serializable neighborhood snapshot centered on node_id. The
result must expose the center identifier, neighboring nodes,
directed relations, and the requested depth without leaking
backend-specific driver objects.
Notes¶
This method defines a pipeline contract, not a storage-engine detail.
Implementations may use Neo4j, in-memory fixtures, or other backends,
but consumers should only rely on the SubGraph semantics.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | |
episteme_pipeline.protocols.graph_store.GraphWriter
¶
Bases: GraphHandle, ABC
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 | |
prune_document_children(document_id, *, keep_chunk_ids, keep_chapter_ids)
abstractmethod
async
¶
Delete the document's Chapter/Chunk nodes that are no longer produced.
Chunk identity is content-addressed, so re-ingesting an edited source
creates new chunk nodes rather than updating the old ones. Callers pass
the ids the current ingest produced; everything else under
document_id is removed. Returns {"chunks": n, "chapters": n}.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | |
upsert_node(label, node_id, properties, *, extra_labels=(), create_only_properties=None)
abstractmethod
async
¶
Idempotently write a node identified by label and node_id.
extra_labels are additional labels stamped on the node (the
:Entity marker, F-05). create_only_properties are written once,
when the node is first created, and never rewritten — first-seen
timestamps belong here so that re-running an unchanged corpus leaves the
graph byte-identical (F-11).
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | |
episteme_pipeline.protocols.graph_store.PhaseCheckpointStore
¶
Bases: GraphHandle, ABC
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 | |
clear_phase_checkpoints(phase)
abstractmethod
async
¶
Clear all item checkpoints for the given phase upon invalidation.
Parameters¶
phase : str Phase identifier tag.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
251 252 253 254 255 256 257 258 259 260 | |
commit_phase_batch(phase, triples, items)
abstractmethod
async
¶
Atomically commit triples and item checkpoints within a single transaction.
Parameters¶
phase : str Phase identifier tag. triples : list[L2Triple] Triples to upsert. items : list[PhaseItemRecord] Phase item checkpoint records to persist.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 | |
filter_unprocessed_items(item_keys, phase)
abstractmethod
async
¶
Filter item keys and return only those not yet marked processed.
Parameters¶
item_keys : list[str] Candidate item keys to check. phase : str Phase identifier tag.
Returns¶
list[str] Subset of item keys that have not been marked in the store.
Source code in packages/episteme-pipeline/episteme_pipeline/protocols/graph_store.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 | |