# Grammar Customization Guide
This document explains how to customize and extend Nasty's grammar rules by creating external grammar resource files.
## Overview
Starting with version 0.2.0, Nasty externalizes grammar rules from hardcoded Elixir modules into configurable `.exs` resource files. This allows you to:
- Customize existing grammar rules without modifying source code
- Create domain-specific grammar variants (e.g., legal, medical, technical)
- Add support for new languages
- A/B test different parsing strategies
- Share grammar rule sets across projects
## Architecture
Grammar rules are stored as Elixir term files (`.exs`) in:
```
priv/languages/{language_code}/grammars/{rule_type}.exs
```
For variants (e.g., formal, informal, technical):
```
priv/languages/{language_code}/variants/{variant_name}/{rule_type}.exs
```
### Language Codes
- English: `en` or `english`
- Spanish: `es` or `spanish`
- Catalan: `ca` or `catalan` (future)
### Rule Types
Each language can have the following grammar rule files:
1. `phrase_rules.exs` - Phrase structure patterns (NP, VP, PP, AdjP, AdvP)
2. `dependency_rules.exs` - Universal Dependencies relations and extraction rules
3. `coordination_rules.exs` - Coordinating conjunctions and coordination patterns
4. `subordination_rules.exs` - Subordinating conjunctions and subordinate clause patterns
## Grammar Loader API
### Loading Grammar Rules
```elixir
alias Nasty.Language.GrammarLoader
# Load default grammar rules
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules)
# Load with variant
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules, variant: "formal")
# Force reload (bypass cache)
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules, force_reload: true)
```
### Cache Management
```elixir
# Clear all cached grammar
GrammarLoader.clear_cache()
# Clear specific cached rules
GrammarLoader.clear_cache(:en, :phrase_rules, :default)
```
### Direct File Loading
```elixir
# Load from custom path
{:ok, rules} = GrammarLoader.load_file("/path/to/custom_rules.exs")
```
## Creating Grammar Files
### File Structure
Grammar files are Elixir term files that evaluate to a map:
```elixir
%{
# Top-level keys define rule categories
rule_category_1: [...],
rule_category_2: %{...},
# Metadata
notes: %{
key: "description"
}
}
```
### Example: Simple Phrase Rules
Create `priv/languages/en/grammars/custom_phrase_rules.exs`:
```elixir
%{
# Noun phrase patterns
noun_phrases: [
# Simple NP: Det + Noun
{:np, [:det, :noun]},
# NP with adjective: Det + Adj + Noun
{:np, [:det, :adj, :noun]},
# NP with PP: Det + Noun + PP
{:np, [:det, :noun, :pp]}
],
# Verb phrase patterns
verb_phrases: [
# Simple VP: just Verb
{:vp, [:verb]},
# VP with object: Verb + NP
{:vp, [:verb, :np]},
# VP with auxiliary: Aux + Verb
{:vp, [:aux, :verb]}
],
notes: %{
version: "1.0.0",
author: "Your Name",
description: "Custom phrase rules for domain-specific parsing"
}
}
```
## English Grammar Reference
### Phrase Rules (`phrase_rules.exs`)
See `priv/languages/en/grammars/phrase_rules.exs` for the complete reference.
Key sections:
```elixir
%{
noun_phrases: [
# List of NP patterns
{:np, [:det, :noun]},
{:np, [:det, :adj, :noun]},
# ...
],
verb_phrases: [
# List of VP patterns
{:vp, [:verb]},
{:vp, [:aux, :verb, :np]},
# ...
],
prepositional_phrases: [
# PP patterns
{:pp, [:prep, :np]},
# ...
],
adjectival_phrases: [
# AdjP patterns
{:adjp, [:adv, :adj]},
# ...
],
adverbial_phrases: [
# AdvP patterns
{:advp, [:adv]},
# ...
],
relative_clauses: [
# Relative clause patterns
{:relative_clause, [:relative_marker, :clause]},
# ...
],
special_rules: [
# Special handling rules
{:comparative_than, :pseudo_prep},
# ...
]
}
```
### Dependency Rules (`dependency_rules.exs`)
See `priv/languages/en/grammars/dependency_rules.exs` for the complete reference.
Key sections:
```elixir
%{
core_arguments: [
# Subject, object, complements
%{
relation: :nsubj,
description: "Nominal subject",
head_pos: [:verb],
dependent_pos: [:noun, :propn, :pron],
example: "The cat sleeps → nsubj(sleeps, cat)"
},
# ...
],
nominal_dependents: [
# Determiners, modifiers
%{relation: :det, ...},
%{relation: :amod, ...},
# ...
],
function_words: [
# Auxiliaries, copulas, markers
%{relation: :aux, ...},
# ...
],
extraction_priorities: [
# Order of dependency extraction
:nsubj, :obj, :det, :amod, # ...
]
}
```
### Coordination Rules (`coordination_rules.exs`)
Key sections:
```elixir
%{
coordinating_conjunctions: [
%{
conjunction: "and",
type: :copulative,
example: "cats and dogs"
},
# ...
],
coordination_patterns: [
%{
pattern: :np_coordination,
structure: "NP CCONJ NP",
example: "cats and dogs"
},
# ...
],
special_cases: [
# Correlative conjunctions, etc.
%{
type: :correlative,
patterns: [
%{pair: ["both", "and"], example: "both cats and dogs"},
# ...
]
}
]
}
```
### Subordination Rules (`subordination_rules.exs`)
Key sections:
```elixir
%{
subordinating_conjunctions: [
%{
conjunction: "because",
type: :causal,
example: "I stayed because it rained"
},
# ...
],
relative_markers: [
%{
marker: "who",
type: :relative_pronoun,
example: "the person who came"
},
# ...
],
subordinate_clause_types: [
%{
type: :adverbial,
dependency_relation: :advcl,
subtypes: [:temporal, :causal, :conditional, ...]
},
# ...
]
}
```
## Spanish Grammar Reference
Spanish grammar files follow the same structure but include Spanish-specific features:
- Post-nominal adjectives: `la casa roja` (the red house)
- Pro-drop: null subjects allowed
- Flexible word order: SVO, VSO, VOS
- Clitic pronouns: `dámelo` (give-me-it)
- Personal 'a': `Veo a Juan` (I see Juan)
- Two copulas: `ser` vs. `estar`
- Phonetic variants: `y`→`e`, `o`→`u` before vowels
See files in `priv/languages/es/grammars/` for complete Spanish grammar.
## Creating Domain-Specific Variants
### Example: Technical English
Create `priv/languages/en/variants/technical/phrase_rules.exs`:
```elixir
%{
# Inherit base rules and add technical-specific patterns
noun_phrases: [
# Standard patterns
{:np, [:det, :noun]},
# Technical compound nouns (e.g., "TCP/IP protocol")
{:np, [:propn, :noun]},
{:np, [:propn, :sym, :propn, :noun]},
# Noun phrases with technical modifiers
{:np, [:num, {:unit, [:noun]}, :noun]}, # "5 GB memory"
# Multi-word technical terms
{:np, [{:many, :noun}]} # "machine learning model"
],
verb_phrases: [
# Standard patterns
{:vp, [:verb, :np]},
# Technical action verbs (instantiate, serialize, etc.)
{:vp, [:tech_verb, :np, :pp]},
# Passive constructions common in technical writing
{:vp, [:aux, :verb, :pp]}
],
notes: %{
domain: "technical",
use_case: "Software documentation, API specs, technical papers"
}
}
```
### Example: Legal English
```elixir
%{
noun_phrases: [
# Legal entities
{:np, [:det, :legal_entity]}, # "the plaintiff", "the defendant"
# Complex legal terms
{:np, [:det, :adj, :legal_term, :pp]}, # "the aforementioned contractual obligation"
# References (Section X, Article Y)
{:np, [:legal_ref_type, :num]} # "Section 5"
],
subordination_patterns: [
# Legal conditionals (provided that, in the event that)
{:conditional, :multiword_legal_conj}
],
notes: %{
domain: "legal",
use_case: "Contracts, legislation, court documents"
}
}
```
## Using Custom Grammar in Code
### Option 1: Load and Use Directly
```elixir
# Load custom grammar
{:ok, custom_phrase_rules} = GrammarLoader.load(:en, :custom_phrase_rules)
# Use in your parser
custom_np_patterns = custom_phrase_rules.noun_phrases
# Process with custom patterns...
```
### Option 2: Extend Parser Module
```elixir
defmodule MyApp.CustomParser do
alias Nasty.Language.GrammarLoader
def parse_technical_text(text) do
# Load technical variant
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules, variant: "technical")
# Parse using custom rules
# ... your parsing logic using rules ...
end
end
```
### Option 3: Runtime Configuration
```elixir
# In config/config.exs
config :nasty,
default_grammar_variant: "technical"
# In your code
variant = Application.get_env(:nasty, :default_grammar_variant, :default)
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules, variant: variant)
```
## Grammar Validation
The grammar loader validates that all files return a map:
```elixir
# Valid
%{
rules: [...],
notes: %{}
}
# Invalid - will raise error
[1, 2, 3] # Not a map
```
For more complex validation, extend `GrammarLoader.validate_rules/1`.
## Best Practices
### 1. Start with Base Grammar
Copy existing grammar files and modify rather than starting from scratch:
```bash
cp priv/languages/en/grammars/phrase_rules.exs \
priv/languages/en/variants/custom/phrase_rules.exs
```
### 2. Document Your Rules
Include comprehensive notes in your grammar files:
```elixir
%{
rules: [...],
notes: %{
version: "1.0.0",
author: "Team Name",
created: "2026-01-08",
description: "Custom grammar for medical text parsing",
changes: [
"Added medical entity patterns",
"Extended VP patterns for medical procedures"
],
examples: [
"The patient underwent cardiac catheterization",
"Diagnose: Type 2 diabetes mellitus"
]
}
}
```
### 3. Test Your Grammar
Create tests for custom grammar:
```elixir
defmodule MyApp.CustomGrammarTest do
use ExUnit.Case
alias Nasty.Language.GrammarLoader
test "custom grammar loads successfully" do
assert {:ok, rules} = GrammarLoader.load(:en, :custom_rules)
assert is_map(rules)
assert Map.has_key?(rules, :noun_phrases)
end
test "custom grammar includes domain patterns" do
{:ok, rules} = GrammarLoader.load(:en, :custom_rules, variant: "medical")
assert Enum.any?(rules.noun_phrases, fn pattern ->
# Check for medical-specific patterns
end)
end
end
```
### 4. Version Your Grammar
Track grammar versions for reproducibility:
```elixir
%{
metadata: %{
version: "2.1.0",
compatible_with: "nasty >= 0.2.0"
},
# ... rules ...
}
```
### 5. Keep Grammar Files Focused
Separate concerns across different rule types:
- Phrase structure → `phrase_rules.exs`
- Dependencies → `dependency_rules.exs`
- Coordination → `coordination_rules.exs`
- Subordination → `subordination_rules.exs`
Don't mix all rules into one file.
## Performance Considerations
### Caching
Grammar files are cached in ETS after first load:
```elixir
# First load: reads from disk
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules) # ~5ms
# Subsequent loads: from cache
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules) # ~0.1ms
```
Clear cache when updating grammar during development:
```elixir
GrammarLoader.clear_cache()
```
### File Size
Keep grammar files under 1MB for fast loading. If needed, split into multiple files:
```
phrase_rules_np.exs # Noun phrase patterns
phrase_rules_vp.exs # Verb phrase patterns
phrase_rules_pp.exs # Prepositional phrase patterns
```
## Troubleshooting
### Grammar File Not Found
```
Grammar file not found: .../en/grammars/missing_rules.exs, using empty rules
```
**Solution**: Check file exists and path is correct. Grammar files must be in `priv/languages/{lang}/grammars/`.
### Invalid Grammar Format
```
** (ArgumentError) Grammar rules must be a map, got: [...]
```
**Solution**: Ensure file evaluates to a map:
```elixir
# Correct
%{rules: [...]}
# Wrong
[...]
```
### Compilation Errors
```
** (SyntaxError) invalid syntax
```
**Solution**: Grammar files must be valid Elixir. Test with:
```bash
elixir priv/languages/en/grammars/your_rules.exs
```
### Cache Issues
If changes to grammar files aren't reflected:
```elixir
# Clear cache
Nasty.Language.GrammarLoader.clear_cache()
# Or force reload
{:ok, rules} = GrammarLoader.load(:en, :phrase_rules, force_reload: true)
```
## Examples Repository
See working examples in the main repository:
- English grammar: `priv/languages/en/grammars/`
- Spanish grammar: `priv/languages/es/grammars/`
- Test fixtures: `test/fixtures/grammars/`
## Contributing Custom Grammars
To contribute grammar variants to the Nasty project:
1. Create grammar files following the structure above
2. Add tests demonstrating the grammar works
3. Document the use case and domain
4. Submit a pull request to the main repository
## Further Reading
- [PARSING_GUIDE.md](PARSING_GUIDE.md) - Understanding the parsing pipeline
- [ENGLISH_GRAMMAR.md](languages/ENGLISH_GRAMMAR.md) - English grammar specification
- [ARCHITECTURE.md](ARCHITECTURE.md) - System architecture overview
- Universal Dependencies: https://universaldependencies.org/
