# ExUtcp

[![Hex.pm](https://img.shields.io/hexpm/v/ex_utcp.svg)](https://hex.pm/packages/ex_utcp)
[![Hex.pm](https://img.shields.io/hexpm/dt/ex_utcp.svg)](https://hex.pm/packages/ex_utcp)
[![Hex.pm](https://img.shields.io/hexpm/l/ex_utcp.svg)](https://hex.pm/packages/ex_utcp)
[![HexDocs.pm](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/ex_utcp)

Elixir implementation of the Universal Tool Calling Protocol (UTCP).

<img width="1024" height="1024" alt="ex_utcp" src="https://github.com/user-attachments/assets/d94d0357-6b73-4e8d-b9d9-a2359131e689" />



## Introduction

The Universal Tool Calling Protocol (UTCP) is a standard for defining and interacting with tools across communication protocols. UTCP emphasizes scalability, interoperability, and ease of use.

Key characteristics:
* **Scalability**: Handles large numbers of tools and providers without performance degradation
* **Interoperability**: Supports multiple provider types including HTTP, [WebSockets](https://tools.ietf.org/html/rfc6455), [gRPC](https://grpc.io/), and CLI tools
* **Ease of Use**: Built on simple, well-defined patterns

## Features

* Transports: HTTP, CLI, WebSocket, gRPC, GraphQL, MCP
* Streaming support across all transports
* OpenAPI Converter: Automatic API discovery and tool generation
* Variable substitution via environment variables or `.env` files
* In-memory repository for providers and tools
* Authentication: API Key, Basic, OAuth2
* Connection pooling and lifecycle management
* Comprehensive test suite with 272+ tests

## Installation

Add `ex_utcp` to your list of dependencies in `mix.exs`:

```elixir
def deps do
  [
    {:ex_utcp, "~> 0.2.0"}
  ]
end
```

## Getting Started

### Basic Usage

```elixir
alias ExUtcp.{Client, Config}

# Create a client configuration
config = Config.new(providers_file_path: "providers.json")

# Start a UTCP client
{:ok, client} = Client.start_link(config)

# Search for tools
{:ok, tools} = Client.search_tools(client, "", 10)

# Call a tool
{:ok, result} = Client.call_tool(client, "provider.tool_name", %{"arg" => "value"})
```

### Programmatic Provider Registration

```elixir
alias ExUtcp.{Client, Config, Providers}

# Create a client
config = Config.new()
{:ok, client} = Client.start_link(config)

# Create an HTTP provider
provider = Providers.new_http_provider([
  name: "my_api",
  url: "https://api.example.com/tools",
  http_method: "POST"
])

# Register the provider
{:ok, tools} = Client.register_tool_provider(client, provider)

# Call a discovered tool
{:ok, result} = Client.call_tool(client, "my_api.echo", %{"message" => "Hello!"})
```

### CLI Provider Example

```elixir
alias ExUtcp.{Client, Config, Providers}

# Create a client
config = Config.new()
{:ok, client} = Client.start_link(config)

# Create a CLI provider
provider = Providers.new_cli_provider([
  name: "my_script",
  command_name: "python my_script.py",
  working_dir: "/path/to/script"
])

# Register the provider
{:ok, tools} = Client.register_tool_provider(client, provider)

# Call a tool
{:ok, result} = Client.call_tool(client, "my_script.greet", %{"name" => "World"})
```

## Configuration

### Provider Configuration File

Create a `providers.json` file to define your providers:

```json
{
  "providers": [
    {
      "name": "http_api",
      "type": "http",
      "http_method": "POST",
      "url": "https://api.example.com/tools",
      "content_type": "application/json",
      "headers": {
        "User-Agent": "ExUtcp/0.2.0"
      },
      "auth": {
        "type": "api_key",
        "api_key": "${API_KEY}",
        "location": "header",
        "var_name": "Authorization"
      }
    },
    {
      "name": "cli_tool",
      "type": "cli",
      "command_name": "python my_tool.py",
      "working_dir": "/opt/tools",
      "env_vars": {
        "PYTHONPATH": "/opt/tools"
      }
    }
  ]
}
```

### Variable Substitution

UTCP supports variable substitution using `${VAR}` or `$VAR` syntax:

```elixir
# Load variables from .env file
{:ok, env_vars} = Config.load_from_env_file(".env")

config = Config.new(
  variables: env_vars,
  providers_file_path: "providers.json"
)
```

## OpenAPI Converter

The OpenAPI Converter automatically discovers and converts OpenAPI specifications into UTCP tools.

### Basic Usage

```elixir
alias ExUtcp.{Client, Config}

# Create a client
{:ok, client} = Client.start_link(%{providers_file_path: nil, variables: %{}})

# Convert OpenAPI spec from URL
{:ok, tools} = Client.convert_openapi(client, "https://api.example.com/openapi.json")

# Convert OpenAPI spec from file
{:ok, tools} = Client.convert_openapi(client, "path/to/spec.yaml")

# Convert with options
{:ok, tools} = Client.convert_openapi(client, spec, %{
  prefix: "my_api",
  auth: %{type: "api_key", api_key: "Bearer ${API_KEY}"}
})
```

### Supported Formats

- OpenAPI 2.0 (Swagger)
- OpenAPI 3.0
- JSON and YAML specifications
- URL and file-based specifications

### Authentication Mapping

The converter automatically maps OpenAPI security schemes to UTCP authentication:

- API Key authentication
- HTTP Basic authentication
- HTTP Bearer authentication
- OAuth2 flows
- OpenID Connect

## Architecture

The library is organized into several main components:

* ExUtcp.Client - Main client interface
* ExUtcp.Config - Configuration management
* ExUtcp.Providers - Provider implementations for different protocols
* ExUtcp.Transports - Transport layer implementations
* ExUtcp.Tools - Tool definitions and management
* ExUtcp.Repository - Tool and provider storage
* ExUtcp.OpenApiConverter - OpenAPI specification conversion

## Implementation Status

### Gap Analysis: UTCP Implementations Comparison

| Feature Category | Python UTCP | Go UTCP | Elixir UTCP | Elixir Coverage |
|------------------|-------------|---------|-------------|-----------------|
| **Core Architecture** | | | | |
| Core Client | Complete | Complete | Complete | 100% |
| Configuration | Complete | Complete | Enhanced | 100% |
| Variable Substitution | Complete | Complete | Complete | 100% |
| **Transports** | | | | |
| HTTP/HTTPS | Complete | Complete | Complete | 100% |
| CLI | Complete | Complete | Complete | 100% |
| WebSocket | Complete | Complete | Complete | 100% |
| gRPC | Complete | Complete | Complete | 100% |
| GraphQL | Complete | Complete | Complete | 100% |
| MCP | Complete | Complete | Complete | 100% |
| SSE | Complete | Complete | Complete | 100% |
| Streamable HTTP | Complete | Complete | Complete | 100% |
| TCP/UDP | Complete | Complete | Not Implemented | 0% |
| WebRTC | Complete | Complete | Not Implemented | 0% |
| **Authentication** | | | | |
| API Key | Complete | Complete | Complete | 100% |
| Basic Auth | Complete | Complete | Complete | 100% |
| OAuth2 | Complete | Complete | Complete | 100% |
| **Advanced Features** | | | | |
| Streaming | Complete | Complete | Complete | 100% |
| Connection Pooling | Complete | Complete | Complete | 100% |
| Error Recovery | Complete | Complete | Complete | 100% |
| OpenAPI Converter | Complete | Complete | Complete | 100% |
| Tool Discovery | Complete | Complete | Complete | 100% |
| Search | Advanced | Advanced | Basic | 60% |
| **Testing** | | | | |
| Unit Tests | Complete | Complete | Complete | 100% |
| Integration Tests | Complete | Complete | Complete | 100% |
| Mock Testing | Complete | Complete | Complete | 100% |
| Test Coverage | High | High | High | 100% |
| **Performance** | | | | |
| Connection Management | Optimized | Optimized | Optimized | 100% |
| Memory Usage | Optimized | Optimized | Optimized | 100% |
| Throughput | High | High | High | 100% |
| **Documentation** | | | | |
| API Docs | Complete | Complete | Complete | 100% |
| Examples | Complete | Complete | Complete | 100% |
| Guides | Complete | Complete | Complete | 100% |

### Priority Recommendations

#### High Priority
- [x] OpenAPI Converter: Automatic API discovery and tool generation
- [ ] Advanced Search: Sophisticated search algorithms
- [ ] TCP/UDP Transport: Low-level network protocols

#### Medium Priority
- [ ] Monitoring: Metrics and health checks
- [ ] Batch Operations: Multiple tool calls
- [ ] WebRTC Transport: Peer-to-peer communication

#### Low Priority
- [ ] Custom Variable Loaders: Beyond .env files
- [ ] API Documentation Generation

### Implementation Status

#### Completed Features
- 6 transports: HTTP, CLI, WebSocket, gRPC, GraphQL, MCP
- Streaming support across all transports
- OpenAPI Converter: Automatic API discovery and tool generation
- Authentication: API Key, Basic, OAuth2
- Connection pooling and lifecycle management
- Error recovery with retry logic
- 272+ tests with comprehensive coverage
- Production examples for all transports

#### Missing Features
- Advanced Search: Sophisticated search algorithms
- TCP/UDP Transport: Low-level network protocols
- WebRTC Transport: Peer-to-peer communication
- Monitoring: Metrics and health checks
- Batch Operations: Multiple tool calls

### Roadmap

#### Phase 1: Core Transports (Completed)
- [x] HTTP/HTTPS, CLI, WebSocket, gRPC, GraphQL, MCP

#### Phase 2: Enhanced Features
- [x] OpenAPI Converter
- [ ] Advanced Search
- [ ] Monitoring and Metrics

#### Phase 3: Extended Protocols
- [ ] TCP/UDP Transport
- [ ] WebRTC Transport

#### Phase 4: Enterprise Features
- [ ] Batch Operations
- [ ] Custom Variable Loaders
- [ ] API Documentation Generation

## Supported Transports

### Implemented
- HTTP/HTTPS: REST API integration with [OpenAPI](https://swagger.io/specification/) support
- CLI: Command-line tool integration
- [WebSocket](https://tools.ietf.org/html/rfc6455): Real-time communication
- [gRPC](https://grpc.io/): High-performance RPC calls with [Protocol Buffers](https://developers.google.com/protocol-buffers)
- [GraphQL](https://graphql.org/): GraphQL API integration with HTTP/HTTPS
- [MCP](https://modelcontextprotocol.io/): Model Context Protocol integration with [JSON-RPC 2.0](https://www.jsonrpc.org/specification)

### Planned
- TCP/UDP: Low-level network protocols
- WebRTC: Peer-to-peer communication

## Examples

See `examples/` directory:
- `http_client.exs` - HTTP provider
- `cli_client.exs` - CLI provider
- `websocket_client.exs` - WebSocket provider
- `grpc_client.exs` - gRPC provider
- `graphql_example.exs` - GraphQL provider
- `mcp_example.exs` - MCP provider
- `streaming_examples.exs` - Streaming examples
- `openapi_example.exs` - OpenAPI Converter examples

## Testing

```bash
# All tests
mix test

# Unit tests only
mix test --exclude integration

# Integration tests only
mix test --only integration
```

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Run the test suite
6. Submit a pull request

## License

MIT License - see [LICENSE](LICENSE) file for details.

## Links

- [UTCP Website](https://www.utcp.io/)
- [Go Implementation](https://github.com/universal-tool-calling-protocol/go-utcp)
- [Python Implementation](https://github.com/universal-tool-calling-protocol/python-utcp)
- [Hex Package](https://hex.pm/packages/ex_utcp)
- [HexDocs](https://hexdocs.pm/ex_utcp)