Compare commits

...
10 Commits
Author SHA1 Message Date
dwrz 10d2db8e75 Rename tool pkg files 2026-01-28 02:26:55 +00:00
dwrz bb85025f81 Add .ignore 2026-01-10 13:37:41 +00:00
dwrz 6b7a30c4ad Rename module to github.com/chimerical-llc/raven
Update go.mod and internal import paths for public release.
2026-01-05 01:29:32 +00:00
dwrz 30e68fd3ee Add .gitignore 2026-01-04 21:12:10 +00:00
dwrz 2e8a85be6f Add README 2026-01-04 21:11:23 +00:00
dwrz 49d41fe31e Add LICENSE (MIT) 2026-01-04 21:11:00 +00:00
dwrz 2ad8a7928d Add example configuration and systemd units 2026-01-04 21:09:05 +00:00
dwrz 0de0793500 Add service orchestration and command
Wire the intake and answer subsystems together into a running
application.

- config: maps YAML file to internal package configuration.
- service: manages lifecycle and graceful shutdown.
- cmd/raven: entry point for flag parsing and signal handling.
2026-01-04 21:07:26 +00:00
dwrz 486cc5fa52 Add answer worker
Subsystem for message processing: parses messages, generates LLM
responses, and replies with SMTP.

Introduces:
- answer: message processing worker.
- llm: OpenAI API compatible client with support for tool execution.
- message: message parsing and response logic.
- tool: converts YAML configuration into executable subprocesses.
- smtp: simple config and client wrapper for sending email.
2026-01-04 21:01:57 +00:00
dwrz c53ee5f6ad Add intake worker
Subsystem to monitor IMAP mailbox for new messages.

Introduces:
- intake: worker that uses IDLE or polling to detect new emails.
- imap: client wrapper for connection management and IMAP commands.
- filter: logic for IMAP search and sender allow-list.
- tracker: concurrency control to prevent processing the same UID twice.
- backoff: for handling connection retries with jitter.
2026-01-04 20:59:26 +00:00
32 changed files with 5972 additions and 2 deletions
+1
View File
@@ -0,0 +1 @@
/bin/
+1
View File
@@ -0,0 +1 @@
vendor/
+7
View File
@@ -0,0 +1,7 @@
Copyright 2026 Chimerical LLC
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+1 -1
View File
@@ -1,4 +1,4 @@
MODULE := raven
MODULE := github.com/chimerical-llc/raven
CMDS := $(notdir $(wildcard ./cmd/*))
BINARIES := $(addprefix bin/,$(CMDS))
CMD ?= raven
+89
View File
@@ -0,0 +1,89 @@
* raven
~raven~ enables querying large language models via email. It retrieves messages from an IMAP mailbox, generates replies using an OpenAI API compatible LLM, and sends them via SMTP. LLMs can optionally invoke user defined external tools.
This project is under active development and breaking changes are possible. The ~go-imap/v2~ dependency is also still in development.
** Features
- Monitor mailboxes with IMAP IDLE or polling
- Query LLMs with an OpenAI API (hosted or local)
- Multimodal input (text and images, inline or attached)
- Configurable concurrency for serializing access to a GPU, or to allow parallel message processing
- Sender allowlist for access control
- Tools defined with YAML, parsed with Go templates, and executed as subprocesses
** Limitations
- ~raven~ can handle replies, but does not attempt to reconstruct the conversation history for the OpenAI API. The quoted body, if provided, is sent alongside the user's latest message.
- Tools do not support variadic arguments. The ~{{ json . }}~ template function may serve as a workaround when a tool accepts a JSON blob.
- Messages may be reprocessed if:
- ~raven~ fails to compose the reply.
- SMTP send fails.
- Marking the message as seen fails.
- The process crashes during processing.
- Reply attribution (='On <date>, <sender> wrote:'=) is a constant string in English.
- Attachments are size-limited per MIME part (~maxPartSize~ = 32MB).
** Security
~raven~ can expose its host to a variety of security risks:
- Denial-of-Service: local LLM inference consumes significant compute; hosted providers incur billing. The volume of email and size of individual emails is another avenue of attack.
- Prompt injection ([[https://genai.owasp.org/llmrisk/llm01-prompt-injection/][direct, indirect, and crescendo attacks]])
- Unexpected costs (from tokens, tools, retries)
- Data exposure, if using a hosted LLM provider, or via manipulated tools.
- Arbitrary command execution and all its consequences, depending on the tools exposed.
The primary mitigation ~raven~ offers is the sender allowlist, which assumes (a) the email provider does not deliver unauthenticated (spoofed) email to the inbox (b) senders can be trusted with access, (c) senders secure access to their accounts.
Other mitigations which a user can consider include:
- Constraining the runtime and sandboxing ~raven~
- Sandboxing tools (jails, containers)
- Opting for narrow tools over broad ones (e.g., scripts versus bash -c).
- Monitoring and constraining resource usage or LLM API costs
- Auditing logs
- Defensive system prompts that constrain model behavior
This software is provided as-is; refer to the license.
** Requirements
- Go 1.25+
- An LLM server with OpenAI-compatible API (e.g., llama.cpp's ~llama-server~)
- IMAP/SMTP email account (or alias)
- IMAP server must support at least two connections.
** Installation
From source:
#+begin_src sh
git clone https://code.chimeric.al/dwrz/raven.git
cd raven
make
#+end_src
The binary is written to ~bin/raven~.
To install to ~$GOBIN~:
#+begin_src sh
make install
#+end_src
** Configuration
Environment variables can be referenced with ~${VAR}~ syntax.
Reference ~config.example.yaml~ for all options.
The path of least resistance is to either use a dedicated email account for ~raven~, or use an alias.
** Deployment
The path to the configuration file is required:
#+begin_src sh
raven -c ~/.config/raven/config.yaml # or /etc/raven/config.yaml
#+end_src
Example ~systemd~ service files for system or user-level services are in ~init/systemd/~.
** License
MIT
+82
View File
@@ -0,0 +1,82 @@
// Command raven monitors an IMAP mailbox for new messages, generates replies
// using an LLM, and sends them via SMTP. See package service for the main
// orchestration logic and package config for configuration options.
package main
import (
"context"
"flag"
"log/slog"
"os"
"runtime/debug"
"github.com/chimerical-llc/raven/internal/config"
"github.com/chimerical-llc/raven/internal/service"
)
func main() {
var cfgPath = flag.String(
"c", "", "path to configuration file (required)",
)
flag.Parse()
var log = slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{
AddSource: true,
Level: slog.LevelDebug,
},
))
if *cfgPath == "" {
log.Error("missing config path")
os.Exit(1)
}
// Log build info.
bi, ok := debug.ReadBuildInfo()
if ok {
var commit, time string
for _, s := range bi.Settings {
switch s.Key {
case "vcs.revision":
commit = s.Value
case "vcs.time":
time = s.Value
}
}
log.Debug(
"raven",
slog.Group(
"build",
slog.String("commit", commit),
slog.String("time", time),
),
)
}
cfg, err := config.Load(*cfgPath)
if err != nil {
log.Error(
"failed to load config",
slog.Any("error", err),
)
os.Exit(1)
}
svc, err := service.New(cfg, log)
if err != nil {
log.Error(
"failed to create service",
slog.Any("error", err),
)
os.Exit(1)
}
if err := svc.Run(context.Background()); err != nil {
log.Error(
"failed to run",
slog.Any("error", err),
)
os.Exit(1)
}
}
+110
View File
@@ -0,0 +1,110 @@
# Example Configuration
# Copy this file to config.yaml and edit as needed.
# Environment variables can be used with ${VAR} syntax.
# Number of concurrent message processing workers (LLM queries + SMTP sends).
# Default: 1 (for llama.cpp without the -np / --parallel flag).
concurrency: 1
# How long to wait for in-flight messages during shutdown.
shutdown_timeout: 30s
# Message filtering configuration.
# All specified filters are AND'd together.
# Excluding allowed_senders, filters are used for IMAP search and will match
# substrings.
filters:
# Only accept emails from these addresses (case-insensitive exact match).
# Empty list = accept from anyone (not recommended for production).
allowed_senders:
- trusted@example.com
# - another@example.com
# Only process emails with body containing these keywords.
# body:
# - "token:e5a78ec2-18f5-4d13-8e97-4062393eb439"
# - "user:e8220e30-cdb6-4c20-a579-35c757bddb7b"
# Only process emails with subject containing this string.
# subject: "[AI]"
# Only process emails sent to this address.
# Useful for plus-addressing: you+ai@example.com
# to: you+ai@example.com
# IMAP server connection for reading messages.
imap:
host: imap.gmail.com
port: "993"
user: you@gmail.com
password: ${RAVEN_IMAP_PASSWORD}
# Message retrieval strategy.
intake:
# Mode: idle | poll
# - idle: Use IMAP IDLE for real-time notifications (requires server support)
# - poll: Periodically check for new messages
mode: poll
# How often to poll for new messages (only used in poll mode).
poll_interval: 30s
# LLM configuration.
llm:
url: http://localhost:8080/v1
key: ${RAVEN_LLM_KEY}
model: local-model
system_prompt: |
You are a helpful assistant responding to plain text emails.
Be concise and helpful. Do not use Markdown formatting.
# Maximum time to wait for LLM response.
timeout: 15m
# SMTP server connection for sending replies.
# User, password, and from default to IMAP values if not specified.
smtp:
host: smtp.gmail.com
port: "587"
# user: you@gmail.com
# password: ${RAVEN_SMTP_PASSWORD}
# from: you@example.com
# External tools available to the LLM.
tools:
- name: echo
description: Echoes back the input message. Useful for testing.
command: echo
arguments:
- "'{{ .message }}'"
parameters:
type: object
properties:
message:
type: string
description: The message to echo back
required:
- message
timeout: 3s
- name: man
description: Read a Unix manual page.
command: man
arguments:
- "{{ .page_name }}"
parameters:
type: object
properties:
page_name:
type: string
description: |
The name of the man page to read.
Can optionally include a section number.
For example: '2 read' or 'cat(1)'.
required:
- page_name
- name: now
description: Retrieves the current local date, time, and timezone.
command: date
arguments:
- "+%Y-%m-%dT%H:%M:%S%:z"
timeout: 1s
+1 -1
View File
@@ -1,4 +1,4 @@
module raven
module github.com/chimerical-llc/raven
go 1.25.5
+22
View File
@@ -0,0 +1,22 @@
[Unit]
Description=raven
Documentation=https://code.chimeric.al/dwrz/raven
After=network.target
[Service]
User=%i
ExecStart=/usr/local/bin/raven -c /etc/raven/config.yaml
Restart=on-failure
RestartSec=10
ProtectSystem=full
PrivateTmp=true
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
NoNewPrivileges=true
StartLimitIntervalSec=60
StartLimitBurst=4
[Install]
WantedBy=multi-user.target
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=raven
Documentation=https://code.chimeric.al/dwrz/raven
After=network.target
[Service]
ExecStart=%h/.local/bin/raven -c %h/.config/raven/config.yaml
Restart=on-failure
RestartSec=10
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
NoNewPrivileges=true
StartLimitIntervalSec=60
StartLimitBurst=4
[Install]
WantedBy=default.target
+263
View File
@@ -0,0 +1,263 @@
// Package answer implements the email response pipeline.
// Workers receive message UIDs, fetch and parse messages, query an LLM
// for responses, and send replies via SMTP.
package answer
import (
"context"
"fmt"
"log/slog"
"time"
"github.com/chimerical-llc/raven/internal/filter"
"github.com/chimerical-llc/raven/internal/imap"
"github.com/chimerical-llc/raven/internal/llm"
"github.com/chimerical-llc/raven/internal/message"
"github.com/chimerical-llc/raven/internal/smtp"
"github.com/chimerical-llc/raven/internal/tracker"
goimap "github.com/emersion/go-imap/v2"
"github.com/emersion/go-message/mail"
)
const fallbackResponse = "Your message was received, but I was unable to generate a response. Please try again later or contact the administrator."
// Worker processes incoming messages through the response pipeline.
// Each worker maintains its own IMAP connection and processes UIDs
// received from a shared work channel. The pipeline:
// 1. Fetch message from IMAP
// 2. Check sender against allowlist
// 3. Query LLM for response
// 4. Send reply via SMTP
// 5. Mark original message as seen
type Worker struct {
// from is the sender address for outgoing replies.
from *mail.Address
// ic is the IMAP client for fetching messages and marking seen.
ic *imap.Client
// id identifies this worker in logs.
id int
// filters determines which senders are allowed.
filters filter.Filters
// llm generates response content.
llm *llm.Client
// log is the worker's logger with worker ID context.
log *slog.Logger
// smtp sends composed replies.
smtp smtp.SMTP
// tracker prevents duplicate processing across workers.
tracker *tracker.Tracker
// work receives UIDs to process.
work <-chan goimap.UID
}
// NewWorker creates a Worker with its own IMAP connection.
// The from address is parsed from smtp.From for reply composition.
func NewWorker(
id int,
filters filter.Filters,
imapConfig imap.Config,
smtp smtp.SMTP,
llm *llm.Client,
tracker *tracker.Tracker,
log *slog.Logger,
work <-chan goimap.UID,
) (*Worker, error) {
ic, err := imap.NewClient(imapConfig, filters, log)
if err != nil {
return nil, fmt.Errorf("create imap client: %v", err)
}
from, err := mail.ParseAddress(smtp.From)
if err != nil {
return nil, fmt.Errorf("parse from: %v", err)
}
return &Worker{
from: from,
ic: ic,
id: id,
filters: filters,
smtp: smtp,
llm: llm,
log: log.With(slog.String(
"worker", fmt.Sprintf("answer[%d]", id),
)),
tracker: tracker,
work: work,
}, nil
}
// Run processes UIDs from the work channel until ctx is canceled
// or the channel is closed. Each UID is processed independently;
// errors are logged but do not stop the worker.
func (w *Worker) Run(ctx context.Context) {
defer w.log.Info("worker stopped", slog.Int("id", w.id))
for {
select {
case <-ctx.Done():
if err := ctx.Err(); err != nil {
w.log.InfoContext(
ctx,
"context closed",
slog.Any("error", err),
)
}
return
case uid, ok := <-w.work:
if !ok {
w.log.InfoContext(
ctx,
"channel closed",
)
return
}
w.respond(ctx, uid)
w.tracker.Release(uid)
}
}
}
// respond handles a single message through the full pipeline.
func (w *Worker) respond(ctx context.Context, uid goimap.UID) {
defer w.ic.Disconnect(ctx)
w.log.InfoContext(ctx, "processing", slog.Any("uid", uid))
// Connect to the IMAP server and retrieve the message.
if err := w.ic.Connect(ctx, w.log, nil); err != nil {
w.log.ErrorContext(
ctx,
"failed to connect IMAP client",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
fetchedMail, err := w.ic.Fetch(uid, true)
if err != nil {
w.log.ErrorContext(
ctx,
"failed to fetch message",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
msg, err := message.New(fetchedMail, w.log)
if err != nil {
w.log.ErrorContext(
ctx,
"skipping: failed to parse message",
slog.Any("error", err),
slog.Any("uid", uid),
)
if err := w.ic.MarkSeen(uid); err != nil {
w.log.ErrorContext(
ctx,
"failed to mark message as seen",
slog.Any("error", err),
slog.Any("uid", uid),
)
}
return
}
// Enforce allowlist.
if !w.filters.MatchSender(msg.Envelope.From[0].Addr()) {
w.log.InfoContext(
ctx,
"skipping: sender not in allowlist",
slog.String("from", msg.Envelope.From[0].Addr()),
slog.Any("uid", uid),
)
if err := w.ic.MarkSeen(uid); err != nil {
w.log.ErrorContext(
ctx,
"failed to mark message as seen",
slog.Any("error", err),
slog.Any("uid", uid),
)
}
return
}
// Disconnect during LLM query to avoid IMAP timeout.
// LLM queries can take significant time depending on model and load.
w.ic.Disconnect(ctx)
res, err := w.llm.Query(ctx, msg)
if err != nil {
w.log.ErrorContext(
ctx,
"failed to query LLM",
slog.Any("error", err),
slog.Any("uid", uid),
)
res = fallbackResponse
}
reply, err := msg.ComposeReply(time.Now(), w.from, res)
if err != nil {
w.log.ErrorContext(
ctx,
"failed to compose reply",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
to, err := reply.Recipients()
if err != nil {
w.log.ErrorContext(
ctx,
"failed to retrieve reply recipients",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
body, err := reply.Bytes()
if err != nil {
w.log.ErrorContext(
ctx,
"failed to retrieve reply body",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
if err := w.smtp.Send(to, body); err != nil {
w.log.ErrorContext(
ctx,
"failed to reply",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
// Reconnect to flag the message as processed.
if err := w.ic.Connect(ctx, w.log, nil); err != nil {
w.log.ErrorContext(
ctx,
"failed to connect IMAP client",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
if err := w.ic.MarkSeen(uid); err != nil {
w.log.ErrorContext(
ctx,
"failed to mark message as seen",
slog.Any("error", err),
slog.Any("uid", uid),
)
return
}
w.log.InfoContext(ctx, "completed", slog.Any("uid", uid))
}
+94
View File
@@ -0,0 +1,94 @@
// Package backoff implements exponential backoff with jitter for retry logic.
package backoff
import (
"fmt"
"math"
"math/rand/v2"
"time"
)
// Config holds the configuration for a Backoff instance.
type Config struct {
// Initial is the starting upper bound for the backoff duration.
// Must be > 0.
Initial time.Duration `yaml:"initial"`
// Max is the absolute maximum upper bound for the backoff duration.
// Must be >= Initial.
Max time.Duration `yaml:"max"`
// RNG is an optional source of randomness.
// If nil, the global math/rand/v2 source is used.
RNG *rand.Rand `yaml:"-"`
}
// Validate checks that the configuration values are sensible.
func (cfg Config) Validate() error {
if cfg.Initial <= 0 {
return fmt.Errorf("invalid initial backoff: %v", cfg.Initial)
}
if cfg.Max < cfg.Initial {
return fmt.Errorf("invalid max duration: %v", cfg.Max)
}
// MaxInt64 will overflow when calculating jitter.
if cfg.Max == math.MaxInt64 {
return fmt.Errorf("max duration cannot be MaxInt64")
}
return nil
}
// Backoff implements exponential backoff with full jitter.
// It is NOT safe for concurrent use.
type Backoff struct {
initial time.Duration
max time.Duration
current time.Duration
rng *rand.Rand
}
// New creates a new Backoff instance with the provided configuration.
func New(cfg Config) (*Backoff, error) {
if err := cfg.Validate(); err != nil {
return nil, fmt.Errorf("invalid config: %v", err)
}
return &Backoff{
initial: cfg.Initial,
max: cfg.Max,
current: cfg.Initial,
rng: cfg.RNG,
}, nil
}
// Next returns the next backoff delay.
func (b *Backoff) Next() time.Duration {
limit := b.current
// Update state for the next call.
if b.current >= b.max {
b.current = b.max
} else if b.current > b.max/2 {
// If doubling would exceed max, just clamp to max.
b.current = b.max
} else {
b.current *= 2
}
// Calculate jitter; return random in [0, limit].
// Int64N(n) returns values in [0, n).
// For [0, limit], we use limit + 1.
var jitter int64
if b.rng != nil {
jitter = b.rng.Int64N(int64(limit) + 1)
} else {
jitter = rand.Int64N(int64(limit) + 1)
}
return time.Duration(jitter)
}
// Reset resets the current backoff cap to the initial value.
// This should be called after a successful operation to restart
// the backoff sequence for future retries.
func (b *Backoff) Reset() {
b.current = b.initial
}
+196
View File
@@ -0,0 +1,196 @@
package backoff
import (
"math"
"math/rand/v2"
"strings"
"testing"
"time"
)
func TestConfigValidate(t *testing.T) {
var tests = []struct {
name string
cfg Config
shouldError bool
}{
{
name: "initial zero",
cfg: Config{Initial: 0, Max: time.Second},
shouldError: true,
},
{
name: "initial negative",
cfg: Config{Initial: -1, Max: time.Second},
shouldError: true,
},
{
name: "max less than initial",
cfg: Config{
Initial: time.Second,
Max: 500 * time.Millisecond,
},
shouldError: true,
},
{
name: "max equals maxint64",
cfg: Config{
Initial: time.Second,
Max: math.MaxInt64,
},
shouldError: true,
},
{
name: "valid",
cfg: Config{
Initial: time.Second,
Max: time.Second,
},
shouldError: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.cfg.Validate()
if tt.shouldError && err == nil {
t.Fatalf("expected error, got nil")
}
if !tt.shouldError && err != nil {
t.Fatalf("unexpected error: %v", err)
}
})
}
}
func TestNewInvalidConfig(t *testing.T) {
_, err := New(Config{Initial: 0, Max: time.Second})
if err == nil {
t.Fatalf("expected error, got nil")
}
if !strings.Contains(err.Error(), "invalid config") {
t.Fatalf("expected invalid config error, got %v", err)
}
}
// TestDistribution uses a fixed seed to check the logic flow and jitter.
func TestDistribution(t *testing.T) {
// RNG with seed (1, 2)
rng := rand.New(rand.NewPCG(1, 2))
initial := 100 * time.Millisecond
maxDur := 400 * time.Millisecond
b, _ := New(Config{
Initial: initial,
Max: maxDur,
RNG: rng,
})
// Generate reference numbers using the same seed to predict outcomes.
refRng := rand.New(rand.NewPCG(1, 2))
expectedCaps := []time.Duration{
100 * time.Millisecond,
200 * time.Millisecond,
400 * time.Millisecond,
400 * time.Millisecond, // Clamped
}
for i, cap := range expectedCaps {
// Expect Next to pick rand(0, cap).
expected := time.Duration(refRng.Int64N(int64(cap) + 1))
got := b.Next()
if got != expected {
t.Errorf(
"step %d: expected %v, got %v (cap was %v)",
i, expected, got, cap,
)
}
if got > cap {
t.Errorf(
"step %d: got %v greater than cap %v",
i, got, cap,
)
}
}
}
// TestOverflowProtection ensures that large Max values do not cause overflow
// during the doubling phase.
func TestOverflowProtection(t *testing.T) {
// Use a Max that is near MaxInt64.
max := time.Duration(math.MaxInt64 - 1)
start := max / 4
b, err := New(Config{
Initial: start,
Max: max,
})
if err != nil {
t.Fatalf("New failed: %v", err)
}
// 1. Current = start.
// Next returns [0, start]. state becomes start*2 (max/2).
_ = b.Next()
// 2. Current = max/2.
// Next returns [0, max/2].
_ = b.Next()
// 3. Current = max.
// Next returns [0, max].
// Logic check: current >= max. state stays max.
val := b.Next()
if val < 0 {
t.Errorf("got negative duration %v, likely overflow", val)
}
// Verify clamp.
// No panic implies success.
for range 5 {
b.Next()
}
}
func TestDefaultRNG(t *testing.T) {
b, err := New(Config{
Initial: time.Millisecond,
Max: 10 * time.Millisecond,
})
if err != nil {
t.Fatalf("New failed: %v", err)
}
got := b.Next()
if got < 0 || got > time.Millisecond {
t.Errorf("out of bounds: %v", got)
}
}
func TestReset(t *testing.T) {
b, _ := New(Config{
Initial: 10 * time.Millisecond,
Max: 100 * time.Millisecond,
})
// Advance state
b.Next() // 10
b.Next() // 20
b.Next() // 40
b.Reset()
// Verify that the cap is reset by checking the bounds of the next
// call. If reset works, the next call is bounded by Initial (10ms).
for range 10 {
got := b.Next()
if got > 10*time.Millisecond {
t.Fatalf(
"call after Reset returned %v; expected <= 10ms",
got,
)
}
b.Reset()
}
}
+151
View File
@@ -0,0 +1,151 @@
// Package config loads and validates YAML configuration for raven.
package config
import (
"fmt"
"os"
"regexp"
"time"
"github.com/chimerical-llc/raven/internal/filter"
"github.com/chimerical-llc/raven/internal/imap"
"github.com/chimerical-llc/raven/internal/intake"
"github.com/chimerical-llc/raven/internal/llm"
"github.com/chimerical-llc/raven/internal/smtp"
"github.com/chimerical-llc/raven/internal/tool"
"gopkg.in/yaml.v3"
)
// Config holds all application configuration sections.
type Config struct {
// Concurrency is the number of concurrent answer workers.
// Defaults to 1.
Concurrency int `yaml:"concurrency"`
// ShutdownTimeout is the maximum time to wait for graceful shutdown.
// Defaults to "30s".
ShutdownTimeout string `yaml:"shutdown_timeout"`
// shutdownTimeout is the parsed duration, set during Load.
shutdownTimeout time.Duration `yaml:"-"`
// Filters configures server-side IMAP search criteria.
Filters filter.Filters `yaml:"filters"`
// IMAP configures the mail server connection for reading messages.
IMAP imap.Config `yaml:"imap"`
// Intake configures the message retrieval strategy (IDLE or poll).
Intake intake.Config `yaml:"intake"`
// LLM configures the language model client.
LLM llm.Config `yaml:"llm"`
// SMTP configures the mail server connection for sending responses.
// Credentials default to IMAP values if not specified.
SMTP smtp.SMTP `yaml:"smtp"`
// Tools defines external commands available to the LLM.
Tools []tool.Tool `yaml:"tools"`
}
// ApplyDefaults sets default values for optional configuration fields.
// Called automatically by Load before validation.
func (cfg *Config) ApplyDefaults() {
if cfg.Concurrency == 0 {
cfg.Concurrency = 1
}
if cfg.ShutdownTimeout == "" {
cfg.ShutdownTimeout = "30s"
}
// Intake
if cfg.Intake.Mode == "" {
cfg.Intake.Mode = intake.ModePoll
}
if cfg.Intake.PollInterval == "" {
cfg.Intake.PollInterval = "30s"
}
// SMTP defaults to IMAP credentials.
if cfg.SMTP.User == "" {
cfg.SMTP.User = cfg.IMAP.User
}
if cfg.SMTP.Password == "" {
cfg.SMTP.Password = cfg.IMAP.Password
}
if cfg.SMTP.From == "" {
cfg.SMTP.From = cfg.IMAP.User
}
}
// GetShutdownTimeout returns the parsed shutdown timeout duration.
func (c *Config) GetShutdownTimeout() time.Duration {
return c.shutdownTimeout
}
// Validate checks that all required fields are present and valid.
// Delegates to each subsection's Validate method.
func (c *Config) Validate() error {
if err := c.IMAP.Validate(); err != nil {
return fmt.Errorf("invalid imap config: %v", err)
}
if err := c.Intake.Validate(); err != nil {
return fmt.Errorf("invalid intake config: %v", err)
}
if err := c.LLM.Validate(); err != nil {
return fmt.Errorf("invalid llm config: %v", err)
}
if err := c.SMTP.Validate(); err != nil {
return fmt.Errorf("invalid smtp config: %v", err)
}
if _, err := time.ParseDuration(c.ShutdownTimeout); err != nil {
return fmt.Errorf(
"invalid shutdown_timeout %q: %w",
c.ShutdownTimeout, err,
)
}
for i, t := range c.Tools {
if err := t.Validate(); err != nil {
return fmt.Errorf("tools[%d] invalid: %v", i, err)
}
}
return nil
}
// Load reads a YAML configuration file, expands environment variables,
// applies defaults, and validates the result. Returns an error if the
// file cannot be read, parsed, or contains invalid configuration.
func Load(path string) (*Config, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read file: %w", err)
}
// Expand environment variables.
// Unset variables are replaced with empty strings.
re := regexp.MustCompile(`\$\{([^}]+)\}`)
expanded := re.ReplaceAllStringFunc(
string(data),
func(match string) string {
// Extract variable name from ${VAR}.
v := match[2 : len(match)-1]
return os.Getenv(v)
},
)
var cfg Config
if err := yaml.Unmarshal([]byte(expanded), &cfg); err != nil {
return nil, fmt.Errorf("parse config: %w", err)
}
cfg.ApplyDefaults()
if err := cfg.Validate(); err != nil {
return nil, fmt.Errorf("invalid config: %v", err)
}
d, err := time.ParseDuration(cfg.ShutdownTimeout)
if err != nil {
return nil, fmt.Errorf(
"parse shutdown timeout: %v", err,
)
}
cfg.shutdownTimeout = d
return &cfg, nil
}
+375
View File
@@ -0,0 +1,375 @@
package config
import (
"os"
"path/filepath"
"testing"
"time"
)
// validConfig returns a minimal valid YAML configuration.
func validConfig() string {
return `
imap:
host: imap.example.com
port: 993
user: test@example.com
password: secret
mailbox: INBOX
intake:
mode: poll
poll_interval: 30s
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
`
}
func writeConfig(t *testing.T, content string) string {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "config.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("write config file: %v", err)
}
return path
}
func TestLoad(t *testing.T) {
tests := []struct {
name string
config string
setup func(t *testing.T)
shouldErr bool
check func(t *testing.T, cfg *Config)
}{
{
name: "valid config with defaults",
config: validConfig(),
check: func(t *testing.T, cfg *Config) {
if cfg.Concurrency != 1 {
t.Errorf(
"Concurrency = %d, want 1",
cfg.Concurrency,
)
}
if cfg.ShutdownTimeout != "30s" {
t.Errorf(
"ShutdownTimeout = %q, want %q",
cfg.ShutdownTimeout, "30s",
)
}
if cfg.GetShutdownTimeout() != 30*time.Second {
t.Errorf(
"GetShutdownTimeout() = %v, want 30s",
cfg.GetShutdownTimeout(),
)
}
if cfg.SMTP.User != cfg.IMAP.User {
t.Errorf(
"SMTP.User = %q, want %q",
cfg.SMTP.User, cfg.IMAP.User,
)
}
if cfg.SMTP.Password != cfg.IMAP.Password {
t.Errorf(
"SMTP.Password not inherited from IMAP",
)
}
if cfg.SMTP.From != cfg.IMAP.User {
t.Errorf(
"SMTP.From = %q, want %q",
cfg.SMTP.From, cfg.IMAP.User,
)
}
if cfg.Intake.Mode != "poll" {
t.Errorf(
"Intake.Mode = %q, want %q",
cfg.Intake.Mode, "poll",
)
}
if cfg.Intake.PollInterval != "30s" {
t.Errorf(
"Intake.PollInterval = %q, want %q",
cfg.Intake.PollInterval, "30s",
)
}
},
},
{
name: "explicit values not overwritten",
config: `
concurrency: 5
shutdown_timeout: "60s"
imap:
host: imap.example.com
port: 993
user: test@example.com
password: secret
mailbox: INBOX
intake:
mode: idle
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
user: smtp-user
password: smtp-password
from: sender@example.com
`,
check: func(t *testing.T, cfg *Config) {
if cfg.Concurrency != 5 {
t.Errorf(
"Concurrency = %d, want 5",
cfg.Concurrency,
)
}
if cfg.ShutdownTimeout != "60s" {
t.Errorf(
"ShutdownTimeout = %q, want %q",
cfg.ShutdownTimeout, "60s",
)
}
if cfg.GetShutdownTimeout() != 60*time.Second {
t.Errorf(
"GetShutdownTimeout() = %v, want 60s",
cfg.GetShutdownTimeout(),
)
}
if cfg.SMTP.User != "smtp-user" {
t.Errorf(
"SMTP.User = %q, want %q",
cfg.SMTP.User, "smtp-user",
)
}
if cfg.SMTP.Password != "smtp-password" {
t.Errorf(
"SMTP.Password = %q, want %q",
cfg.SMTP.Password,
"smtp-password",
)
}
if cfg.SMTP.From != "sender@example.com" {
t.Errorf(
"SMTP.From = %q, want %q",
cfg.SMTP.From,
"sender@example.com",
)
}
},
},
{
name: "env expansion",
config: `
imap:
host: imap.example.com
port: 993
user: test@example.com
password: ${TEST_IMAP_PASSWORD}
mailbox: INBOX
intake:
mode: poll
poll_interval: 30s
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
`,
setup: func(t *testing.T) {
t.Setenv("TEST_IMAP_PASSWORD", "env-secret")
},
check: func(t *testing.T, cfg *Config) {
if cfg.IMAP.Password != "env-secret" {
t.Errorf(
"IMAP.Password = %q, want %q",
cfg.IMAP.Password,
"env-secret",
)
}
},
},
{
name: "env expansion unset var becomes empty",
config: `
imap:
host: imap.example.com
port: 993
user: test@example.com
password: ${TEST_UNSET_VAR}
mailbox: INBOX
intake:
mode: poll
poll_interval: 30s
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
`,
setup: func(t *testing.T) {
t.Setenv("TEST_UNSET_VAR", "")
},
shouldErr: true,
},
{
name: "invalid yaml",
config: `imap: [invalid yaml`,
shouldErr: true,
},
{
name: "invalid shutdown_timeout",
config: `
imap:
host: imap.example.com
port: 993
user: test@example.com
password: secret
mailbox: INBOX
intake:
mode: poll
poll_interval: 30s
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
shutdown_timeout: "not-a-duration"
`,
shouldErr: true,
},
{
name: "config with tools",
config: `
imap:
host: imap.example.com
port: 993
user: test@example.com
password: secret
mailbox: INBOX
intake:
mode: poll
poll_interval: 30s
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
tools:
- name: echo
description: echoes input
command: echo
`,
check: func(t *testing.T, cfg *Config) {
if len(cfg.Tools) != 1 {
t.Errorf(
"len(Tools) = %d, want 1",
len(cfg.Tools),
)
}
if cfg.Tools[0].Name != "echo" {
t.Errorf(
"Tools[0].Name = %q, want %q",
cfg.Tools[0].Name, "echo",
)
}
},
},
{
name: "intake mode defaults when empty",
config: `
imap:
host: imap.example.com
port: 993
user: test@example.com
password: secret
mailbox: INBOX
llm:
provider: llama.cpp
model: qwen3-4b
url: http://localhost:8080
smtp:
host: smtp.example.com
port: 587
`,
check: func(t *testing.T, cfg *Config) {
if cfg.Intake.Mode != "poll" {
t.Errorf(
"Intake.Mode = %q, want %q",
cfg.Intake.Mode, "poll",
)
}
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.setup != nil {
tt.setup(t)
}
path := writeConfig(t, tt.config)
cfg, err := Load(path)
if tt.shouldErr {
if err == nil {
t.Fatal("Load() expected error")
}
return
}
if err != nil {
t.Fatalf("Load() error = %v", err)
}
if tt.check != nil {
tt.check(t, cfg)
}
})
}
}
func TestLoad_FileNotFound(t *testing.T) {
if _, err := Load("/nonexistent/path/config.yaml"); err == nil {
t.Fatal("Load() expected error for missing file")
}
}
+52
View File
@@ -0,0 +1,52 @@
// Package filter provides message filtering configuration.
// It is used to restrict which messages are processed by the answer workers.
//
// NOTE: IMAP Search Limitations
// IMAP SEARCH (RFC 3501 §6.4.4) uses case-insensitive substring matching for
// text fields.
// A filter like From: "alice@example.com" would match:
// - alice@example.com
// - malice@example.com
// - alice@example.com.evil.org
//
// For this reason, server-side IMAP filters should be considered
// performance optimizations only, not security controls. Use the
// [Filters.Senders] allowlist for exact sender matching.
//
// [RFC 3501 §6.4.4]:
// https://datatracker.ietf.org/doc/html/rfc3501#section-6.4.4
package filter
import "strings"
// Filters holds message filtering configuration for IMAP searches
// and sender verification.
type Filters struct {
// Body contains keywords that must appear in the message body.
Body []string `yaml:"body"`
// From filters messages by the From header field.
From string `yaml:"from"`
// Senders is an allowlist of email addresses permitted to receive
// replies. If empty, all senders are allowed.
Senders []string `yaml:"allowed_senders"`
// Subject filters messages by the Subject header field.
Subject string `yaml:"subject"`
// To filters messages by the To header field.
To string `yaml:"to"`
}
// MatchSender checks if the sender is in the allowed list.
// Returns true if the allowlist is empty (allow all) or if the sender matches.
// Comparison is case-insensitive and ignores leading/trailing whitespace.
func (f *Filters) MatchSender(sender string) bool {
if len(f.Senders) == 0 {
return true
}
sender = strings.ToLower(strings.TrimSpace(sender))
for _, allowed := range f.Senders {
if strings.ToLower(strings.TrimSpace(allowed)) == sender {
return true
}
}
return false
}
+91
View File
@@ -0,0 +1,91 @@
package filter
import "testing"
func TestMatchSender(t *testing.T) {
tests := []struct {
name string
senders []string
sender string
want bool
}{
{
name: "empty allowlist allows all",
senders: nil,
sender: "anyone@example.com",
want: true,
},
{
name: "exact match",
senders: []string{"allowed@example.com"},
sender: "allowed@example.com",
want: true,
},
{
name: "case insensitive match",
senders: []string{"Allowed@Example.COM"},
sender: "allowed@example.com",
want: true,
},
{
name: "sender case insensitive",
senders: []string{"allowed@example.com"},
sender: "ALLOWED@EXAMPLE.COM",
want: true,
},
{
name: "whitespace trimmed from sender",
senders: []string{"allowed@example.com"},
sender: " allowed@example.com ",
want: true,
},
{
name: "whitespace trimmed from allowlist",
senders: []string{" allowed@example.com "},
sender: "allowed@example.com",
want: true,
},
{
name: "not in allowlist",
senders: []string{"allowed@example.com"},
sender: "blocked@example.com",
want: false,
},
{
name: "multiple allowed senders",
senders: []string{"one@example.com", "two@example.com", "three@example.com"},
sender: "two@example.com",
want: true,
},
{
name: "partial match not allowed",
senders: []string{"allowed@example.com"},
sender: "allowed@example",
want: false,
},
{
name: "empty sender with allowlist",
senders: []string{"allowed@example.com"},
sender: "",
want: false,
},
{
name: "empty sender with empty allowlist",
senders: nil,
sender: "",
want: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
f := &Filters{Senders: tt.senders}
if got := f.MatchSender(tt.sender); got != tt.want {
t.Errorf(
"MatchSender(%q) = %v, want %v",
tt.sender, got, tt.want,
)
}
})
}
}
+264
View File
@@ -0,0 +1,264 @@
// Package imap provides an IMAP client wrapper for monitoring and fetching
// email messages. It handles connection management, IDLE support for push
// notifications, and message operations like fetching and marking as seen.
//
// The client supports server-side filtering via [filter.Filters] to limit
// which messages are returned by [Client.Unseen].
package imap
import (
"context"
"fmt"
"log/slog"
"time"
"github.com/chimerical-llc/raven/internal/filter"
"github.com/emersion/go-imap/v2"
"github.com/emersion/go-imap/v2/imapclient"
)
// Config holds IMAP server connection parameters.
type Config struct {
// Host is the IMAP server hostname.
Host string `yaml:"host"`
// Password is the authentication password or app-specific password.
Password string `yaml:"password"`
// Port is the IMAP server port (typically "993" for TLS).
Port string `yaml:"port"`
// User is the authentication username, usually an email address.
User string `yaml:"user"`
}
// Address returns the host:port string for dialing.
func (c *Config) Address() string {
return fmt.Sprintf("%s:%s", c.Host, c.Port)
}
// Validate checks that all required fields are present.
func (c *Config) Validate() error {
if c.Host == "" {
return fmt.Errorf("missing host")
}
if c.Password == "" {
return fmt.Errorf("missing password")
}
if c.Port == "" {
return fmt.Errorf("missing port")
}
if c.User == "" {
return fmt.Errorf("missing user")
}
return nil
}
// Client wraps an IMAP client with connection management and filtering.
type Client struct {
cfg Config
filters filter.Filters
client *imapclient.Client
log *slog.Logger
}
// NewClient creates a new Client with the given configuration.
// The client is not connected until [Client.Connect] is called.
func NewClient(
cfg Config,
filters filter.Filters,
log *slog.Logger,
) (*Client, error) {
if err := cfg.Validate(); err != nil {
return nil, fmt.Errorf("invalid config: %v", err)
}
return &Client{
cfg: cfg,
filters: filters,
log: log,
}, nil
}
// Client returns the underlying imapclient.Client.
// Returns nil if not connected.
func (c *Client) Client() *imapclient.Client {
return c.client
}
// Connect establishes a TLS connection to the IMAP server,
// authenticates, and selects the INBOX.
func (c *Client) Connect(
ctx context.Context,
log *slog.Logger,
opts *imapclient.Options,
) error {
c.log = log
c.log.InfoContext(ctx, "connecting to IMAP server")
client, err := imapclient.DialTLS(c.cfg.Address(), opts)
if err != nil {
return fmt.Errorf("dial TLS: %w", err)
}
c.client = client
c.log.InfoContext(ctx, "connected to IMAP server")
c.log.InfoContext(ctx, "logging into to IMAP server")
if err := client.Login(c.cfg.User, c.cfg.Password).Wait(); err != nil {
return fmt.Errorf("login: %w", err)
}
c.log.InfoContext(ctx, "logged into IMAP server")
c.log.InfoContext(ctx, "selecting INBOX")
if _, err := client.Select("INBOX", nil).Wait(); err != nil {
return fmt.Errorf("select: %w", err)
}
c.log.InfoContext(ctx, "selected INBOX")
return nil
}
// CanIdle reports whether the server advertises IDLE capability (RFC 2177).
func (c *Client) CanIdle() bool {
return c.client.Caps().Has(imap.CapIdle)
}
// Disconnect logs out from the server and closes the connection.
// Safe to call on a nil or disconnected client.
func (c *Client) Disconnect(ctx context.Context) {
if c.client == nil {
return
}
c.log.InfoContext(ctx, "logging out")
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
var done = make(chan error, 1)
go func() {
done <- c.client.Logout().Wait()
}()
select {
case <-ctx.Done():
c.log.InfoContext(
ctx,
"context closed",
slog.Any("reason", ctx.Err()),
)
case err := <-done:
if err != nil {
c.log.Warn(
"failed to logout from IMAP server",
slog.Any("error", err),
)
}
}
c.client.Close()
c.client = nil
}
// Fetch retrieves a message by UID. If peek is true, the message is
// fetched without marking it as seen (using BODY.PEEK[]).
func (c *Client) Fetch(uid imap.UID, peek bool) (
*imapclient.FetchMessageBuffer, error,
) {
if c.client == nil {
return nil, fmt.Errorf("client not connected")
}
opts := &imap.FetchOptions{Envelope: true, UID: true}
if peek {
opts.BodySection = []*imap.FetchItemBodySection{{Peek: true}}
}
res, err := c.client.Fetch(imap.UIDSetNum(uid), opts).Collect()
if err != nil {
return nil, fmt.Errorf("fetch: %w", err)
}
if len(res) == 0 {
return nil, fmt.Errorf("message not found")
}
return res[0], nil
}
// Idle starts an IDLE command (RFC 2177) to wait for server notifications.
// The caller must call Close on the returned command to end IDLE mode.
func (c *Client) Idle() (*imapclient.IdleCommand, error) {
return c.client.Idle()
}
// MarkSeen adds the \Seen flag to the message with the given UID.
func (c *Client) MarkSeen(uid imap.UID) error {
if c.client == nil {
return fmt.Errorf("client not connected")
}
if _, err := c.client.Store(
imap.UIDSetNum(uid),
&imap.StoreFlags{
Op: imap.StoreFlagsAdd,
Silent: true,
Flags: []imap.Flag{imap.FlagSeen},
},
nil,
).Collect(); err != nil {
return fmt.Errorf("collect: %w", err)
}
return nil
}
// Noop sends a NOOP command to refresh mailbox state.
func (c *Client) Noop() error {
if c.client == nil {
return fmt.Errorf("client not connected")
}
return c.client.Noop().Wait()
}
// Unseen returns UIDs of messages matching the client's filters that
// do not have the \Seen flag. Filters are applied server-side.
func (c *Client) Unseen(ctx context.Context) ([]imap.UID, error) {
sc := &imap.SearchCriteria{
NotFlag: []imap.Flag{imap.FlagSeen},
}
if len(c.filters.Body) > 0 {
sc.Body = c.filters.Body
}
if c.filters.From != "" {
sc.Header = append(
sc.Header,
imap.SearchCriteriaHeaderField{
Key: "From",
Value: c.filters.From,
},
)
}
if c.filters.Subject != "" {
sc.Header = append(
sc.Header,
imap.SearchCriteriaHeaderField{
Key: "Subject",
Value: c.filters.Subject,
},
)
}
if c.filters.To != "" {
sc.Header = append(
sc.Header,
imap.SearchCriteriaHeaderField{
Key: "To",
Value: c.filters.To,
},
)
}
res, err := c.client.UIDSearch(sc, nil).Wait()
if err != nil {
return nil, fmt.Errorf("search: %v", err)
}
uids := res.AllUIDs()
return uids, nil
}
+224
View File
@@ -0,0 +1,224 @@
package imap
import (
"log/slog"
"testing"
"github.com/chimerical-llc/raven/internal/filter"
)
func TestConfigAddress(t *testing.T) {
tests := []struct {
name string
cfg Config
want string
}{
{
name: "standard config",
cfg: Config{Host: "imap.example.com", Port: "993"},
want: "imap.example.com:993",
},
{
name: "empty host",
cfg: Config{Host: "", Port: "993"},
want: ":993",
},
{
name: "empty port",
cfg: Config{Host: "imap.example.com", Port: ""},
want: "imap.example.com:",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := tt.cfg.Address(); got != tt.want {
t.Errorf(
"Address() = %q, want %q",
got, tt.want,
)
}
})
}
}
func TestConfigValidate(t *testing.T) {
tests := []struct {
name string
cfg Config
wantErr string
}{
{
name: "valid config",
cfg: Config{
Host: "imap.example.com",
Password: "secret",
Port: "993",
User: "user@example.com",
},
wantErr: "",
},
{
name: "missing host",
cfg: Config{
Password: "secret",
Port: "993",
User: "user@example.com",
},
wantErr: "missing host",
},
{
name: "missing password",
cfg: Config{
Host: "imap.example.com",
Port: "993",
User: "user@example.com",
},
wantErr: "missing password",
},
{
name: "missing port",
cfg: Config{
Host: "imap.example.com",
Password: "secret",
User: "user@example.com",
},
wantErr: "missing port",
},
{
name: "missing user",
cfg: Config{
Host: "imap.example.com",
Password: "secret",
Port: "993",
},
wantErr: "missing user",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.cfg.Validate()
if tt.wantErr == "" {
if err != nil {
t.Errorf(
"Validate() unexpected error: %v",
err,
)
}
return
}
if err == nil {
t.Errorf(
"Validate() expected error containing %q, got nil",
tt.wantErr,
)
return
}
if got := err.Error(); got != tt.wantErr {
t.Errorf(
"Validate() error = %q, want %q",
got, tt.wantErr,
)
}
})
}
}
func TestNewClient(t *testing.T) {
validCfg := Config{
Host: "imap.example.com",
Password: "secret",
Port: "993",
User: "user@example.com",
}
tests := []struct {
name string
cfg Config
filters filter.Filters
wantErr bool
}{
{
name: "valid config",
cfg: validCfg,
filters: filter.Filters{},
wantErr: false,
},
{
name: "valid config with filters",
cfg: validCfg,
filters: filter.Filters{
Subject: "test",
To: "recipient@example.com",
Body: []string{"keyword"},
},
wantErr: false,
},
{
name: "invalid config",
cfg: Config{Host: "imap.example.com"},
filters: filter.Filters{},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
client, err := NewClient(
tt.cfg, tt.filters, slog.Default(),
)
if tt.wantErr {
if err == nil {
t.Error(
"NewClient() expected error, got nil",
)
}
return
}
if err != nil {
t.Errorf(
"NewClient() unexpected error: %v", err,
)
return
}
if client == nil {
t.Error("NewClient() returned nil client")
return
}
// Verify client is not connected.
if client.Client() != nil {
t.Error(
"NewClient() client should not be connected",
)
}
})
}
}
func TestClientNotConnected(t *testing.T) {
cfg := Config{
Host: "imap.example.com",
Password: "secret",
Port: "993",
User: "user@example.com",
}
client, err := NewClient(cfg, filter.Filters{}, slog.Default())
if err != nil {
t.Fatalf("NewClient() unexpected error: %v", err)
}
t.Run("Fetch without connection", func(t *testing.T) {
_, err := client.Fetch(1, true)
if err == nil {
t.Error("Fetch() expected error when not connected")
}
})
t.Run("MarkSeen without connection", func(t *testing.T) {
err := client.MarkSeen(1)
if err == nil {
t.Error("MarkSeen() expected error when not connected")
}
})
}
+398
View File
@@ -0,0 +1,398 @@
// Package intake monitors an IMAP mailbox for new messages and dispatches
// them to answer workers for processing. It supports two modes: IDLE for
// servers with real-time push notifications, and poll for periodic checking.
package intake
import (
"context"
"fmt"
"log/slog"
"slices"
"time"
"github.com/chimerical-llc/raven/internal/backoff"
"github.com/chimerical-llc/raven/internal/filter"
"github.com/chimerical-llc/raven/internal/imap"
"github.com/chimerical-llc/raven/internal/tracker"
goimap "github.com/emersion/go-imap/v2"
"github.com/emersion/go-imap/v2/imapclient"
)
// Config holds intake worker settings.
type Config struct {
// Mode selects IDLE or poll-based message retrieval.
Mode Mode `yaml:"mode"`
// PollInterval is the duration between checks in poll mode.
// Ignored in IDLE mode. Examples: "5s", "1m", "24h".
PollInterval string `yaml:"poll_interval"`
}
// Validate checks that the configuration values are sensible.
func (c *Config) Validate() error {
if !c.Mode.Valid() {
return fmt.Errorf("invalid mode")
}
d, err := time.ParseDuration(c.PollInterval)
if d == 0 || err != nil {
return fmt.Errorf(
"invalid poll_interval %q: %v",
c.PollInterval, err,
)
}
return nil
}
// Mode represents the message retrieval strategy.
type Mode string
// Valid returns true if the mode is a recognized value.
func (m Mode) Valid() bool {
return slices.Contains([]Mode{ModeIdle, ModePoll}, m)
}
// String returns the mode as a string.
func (m Mode) String() string {
return string(m)
}
const (
// ModeIdle uses IMAP IDLE for real-time notifications.
ModeIdle Mode = "idle"
// ModePoll uses periodic polling at a configured interval.
ModePoll Mode = "poll"
)
// Worker monitors an IMAP mailbox and dispatches message UIDs to answer
// workers. It maintains a persistent connection with automatic reconnection
// and exponential backoff on errors.
type Worker struct {
// backoff controls retry delays after connection failures.
backoff *backoff.Backoff
// cfg holds the worker configuration.
cfg Config
// ic is the IMAP client for mailbox operations.
ic *imap.Client
// interval is the parsed poll interval duration.
interval time.Duration
// log is the worker's logger with worker context.
log *slog.Logger
// tracker prevents duplicate processing across workers.
tracker *tracker.Tracker
// update signals new messages during IDLE.
update chan struct{}
// work sends fetched message UIDs to answer workers.
work chan<- goimap.UID
}
// NewWorker creates an intake Worker with the provided configuration.
// The tracker coordinates with answer workers to prevent duplicate processing.
// The work channel receives UIDs for messages that need responses.
func NewWorker(
cfg Config,
filters filter.Filters,
imapConfig imap.Config,
log *slog.Logger,
tracker *tracker.Tracker,
work chan<- goimap.UID,
) (*Worker, error) {
if err := cfg.Validate(); err != nil {
return nil, fmt.Errorf("invalid config: %v", err)
}
if tracker == nil {
return nil, fmt.Errorf("missing tracker")
}
if work == nil {
return nil, fmt.Errorf("missing work channel")
}
b, err := backoff.New(backoff.Config{
Initial: time.Second,
Max: time.Minute,
})
if err != nil {
return nil, fmt.Errorf("create backoff: %v", err)
}
ic, err := imap.NewClient(imapConfig, filters, log)
if err != nil {
return nil, fmt.Errorf("create imap client: %v", err)
}
var w = &Worker{
backoff: b,
cfg: cfg,
ic: ic,
log: log.With(slog.String("worker", "intake")),
tracker: tracker,
work: work,
}
switch w.cfg.Mode {
case ModeIdle:
w.update = make(chan struct{}, 1)
case ModePoll:
d, err := time.ParseDuration(cfg.PollInterval)
if err != nil {
return nil, fmt.Errorf(
"parse poll interval: %v", err,
)
}
w.interval = d
}
return w, nil
}
// Run starts the intake worker. It connects to the IMAP server and monitors
// for new messages using either IDLE or poll mode based on configuration.
// Handles reconnection automatically with exponential backoff on errors.
// Returns nil when the context is canceled.
func (w *Worker) Run(ctx context.Context) error {
w.log.InfoContext(ctx, "running intake worker")
defer w.log.InfoContext(ctx, "intake worker terminating")
defer w.ic.Disconnect(ctx)
// For IDLE mode, set up handler to signal new messages.
var opts *imapclient.Options
if w.cfg.Mode == ModeIdle {
opts = &imapclient.Options{
UnilateralDataHandler: &imapclient.UnilateralDataHandler{
Mailbox: func(data *imapclient.UnilateralDataMailbox) {
if data.NumMessages != nil {
select {
case w.update <- struct{}{}:
default:
}
}
},
},
}
}
for {
// Context closed -- terminate.
if err := ctx.Err(); err != nil {
w.log.InfoContext(
ctx,
"context closed",
slog.Any("reason", ctx.Err()),
)
return nil
}
// Attempt to connect.
if w.ic.Client() == nil {
if err := w.ic.Connect(
ctx, w.log, opts,
); err != nil {
// Failed to connect; backoff and try again.
w.log.ErrorContext(
ctx,
"failed to connect to IMAP server",
slog.Any("error", err),
)
wait := w.backoff.Next()
w.log.InfoContext(
ctx,
"retrying after backoff",
slog.Any("wait", wait),
)
// Wait for backoff, unless context closed.
select {
case <-ctx.Done():
continue
case <-time.After(wait):
continue
}
}
// Connected.
// For IDLE mode, verify server capability.
if w.cfg.Mode == ModeIdle && !w.ic.CanIdle() {
return fmt.Errorf(
"server lacks IDLE capability",
)
}
}
// Retrieve messages with the appropriate mode.
var err error
switch w.cfg.Mode {
case ModeIdle:
err = w.idle(ctx)
case ModePoll:
err = w.poll(ctx)
default:
return fmt.Errorf("unrecognized mode: %q", w.cfg.Mode)
}
if err != nil {
// Diagnose the error.
// Context canceled: loop, log, return.
if err := ctx.Err(); err != nil {
continue
}
// Otherwise, error on intake or idle.
w.log.ErrorContext(
ctx,
"IMAP error",
slog.Any("error", err),
)
wait := w.backoff.Next()
w.log.InfoContext(
ctx,
"retrying after backoff",
slog.Any("wait", wait),
)
select {
case <-ctx.Done():
continue
case <-time.After(wait):
}
// Force reconnect on next loop.
w.ic.Disconnect(ctx)
}
}
}
// poll runs the polling loop, checking for new messages at the configured
// interval. The interval could be 5 seconds or 24 hours depending on use case.
// Attempts to reuse the established connection, but the caller handles
// reconnection if the server closes an idle connection.
func (w *Worker) poll(ctx context.Context) error {
for {
// Refresh mailbox state before searching.
if err := w.ic.Noop(); err != nil {
return fmt.Errorf("noop: %v", err)
}
if err := w.intake(ctx); err != nil {
return fmt.Errorf("intake: %v", err)
}
// Connection is healthy. Reset backoff.
w.backoff.Reset()
select {
case <-ctx.Done():
return fmt.Errorf("context closed: %v", ctx.Err())
case <-time.After(w.interval):
continue
}
}
}
// idle runs the IDLE loop, using IMAP IDLE for real-time message notifications.
// After each intake cycle, issues an IDLE command and waits for the server
// to signal new messages. Handles IDLE termination and reconnection.
func (w *Worker) idle(ctx context.Context) error {
for {
if err := w.intake(ctx); err != nil {
return fmt.Errorf("intake: %v", err)
}
// Connection is healthy. Reset backoff.
w.backoff.Reset()
// Issue the IDLE command.
w.log.InfoContext(ctx, "entering IDLE mode")
idleCmd, err := w.ic.Idle()
if err != nil {
return fmt.Errorf("idle: %v", err)
}
// Monitor the IDLE command.
idleDone := make(chan error, 1)
go func() {
idleDone <- idleCmd.Wait()
}()
// Wait for: shutdown, connection death, or new message.
select {
case <-ctx.Done():
if err := idleCmd.Close(); err != nil {
w.log.WarnContext(
ctx,
"failed to close IDLE command",
slog.Any("error", err),
)
}
return fmt.Errorf("context closed: %v", ctx.Err())
case err := <-idleDone:
// Connection died or IDLE ended unexpectedly.
if err != nil {
return fmt.Errorf("idle wait: %v", err)
}
// IDLE ended without error (server closed).
w.log.InfoContext(ctx, "IDLE ended; refreshing")
return nil
case <-w.update:
w.log.InfoContext(ctx, "IDLE: new message received")
if err := idleCmd.Close(); err != nil {
<-idleDone
return fmt.Errorf("idle close: %v", err)
}
<-idleDone
}
}
}
// intake fetches unseen messages and sends their UIDs to the work channel.
// Uses the tracker to skip messages already being processed by answer workers.
// Returns early if the work queue is full, deferring remaining messages to
// the next cycle.
func (w *Worker) intake(ctx context.Context) error {
w.log.InfoContext(ctx, "fetching unseen messages")
uids, err := w.ic.Unseen(ctx)
if err != nil {
return fmt.Errorf("retrieve unseen messages: %v", err)
}
if len(uids) == 0 {
w.log.InfoContext(ctx, "no new messages")
return nil
}
w.log.InfoContext(
ctx,
"found unseen messages",
slog.Int("count", len(uids)),
)
for _, uid := range uids {
if !w.tracker.TryAcquire(uid) {
w.log.InfoContext(
ctx,
"skipping message; already acquired",
slog.Any("uid", uid),
)
continue
}
select {
case <-ctx.Done():
w.tracker.Release(uid)
w.log.InfoContext(
ctx,
"context closed",
slog.Any("reason", ctx.Err()),
)
return nil
case w.work <- uid:
w.log.InfoContext(
ctx, "enqueued", slog.Any("uid", uid),
)
default:
// Queue full, release and defer to next cycle.
w.tracker.Release(uid)
w.log.InfoContext(
ctx, "work queue full, deferring remaining",
)
return nil
}
}
return nil
}
+210
View File
@@ -0,0 +1,210 @@
package intake
import (
"strings"
"testing"
"github.com/chimerical-llc/raven/internal/filter"
"github.com/chimerical-llc/raven/internal/imap"
"github.com/chimerical-llc/raven/internal/tracker"
goimap "github.com/emersion/go-imap/v2"
)
func TestConfigValidate(t *testing.T) {
tests := []struct {
name string
cfg Config
wantErr bool
}{
{
name: "valid idle mode",
cfg: Config{
Mode: ModeIdle,
PollInterval: "30s",
},
wantErr: false,
},
{
name: "valid poll mode",
cfg: Config{
Mode: ModePoll,
PollInterval: "5m",
},
wantErr: false,
},
{
name: "valid poll mode with long interval",
cfg: Config{
Mode: ModePoll,
PollInterval: "24h",
},
wantErr: false,
},
{
name: "invalid mode",
cfg: Config{
Mode: "invalid",
PollInterval: "30s",
},
wantErr: true,
},
{
name: "empty mode",
cfg: Config{
Mode: "",
PollInterval: "30s",
},
wantErr: true,
},
{
name: "missing poll interval",
cfg: Config{
Mode: ModeIdle,
PollInterval: "",
},
wantErr: true,
},
{
name: "invalid poll interval",
cfg: Config{
Mode: ModePoll,
PollInterval: "invalid",
},
wantErr: true,
},
{
name: "zero poll interval",
cfg: Config{
Mode: ModePoll,
PollInterval: "0s",
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.cfg.Validate()
if (err != nil) != tt.wantErr {
t.Errorf(
"Validate() error = %v, wantErr %v",
err, tt.wantErr,
)
}
})
}
}
func TestModeValid(t *testing.T) {
tests := []struct {
mode Mode
want bool
}{
{ModeIdle, true},
{ModePoll, true},
{"idle", true},
{"poll", true},
{"", false},
{"invalid", false},
{"IDLE", false},
{"POLL", false},
}
for _, tt := range tests {
t.Run(string(tt.mode), func(t *testing.T) {
if got := tt.mode.Valid(); got != tt.want {
t.Errorf(
"Mode(%q).Valid() = %v, want %v",
tt.mode, got, tt.want,
)
}
})
}
}
func TestModeString(t *testing.T) {
tests := []struct {
mode Mode
want string
}{
{ModeIdle, "idle"},
{ModePoll, "poll"},
{"custom", "custom"},
}
for _, tt := range tests {
t.Run(tt.want, func(t *testing.T) {
if got := tt.mode.String(); got != tt.want {
t.Errorf(
"Mode.String() = %v, want %v",
got, tt.want,
)
}
})
}
}
func TestNewWorkerValidation(t *testing.T) {
validConfig := Config{
Mode: ModeIdle,
PollInterval: "30s",
}
validTracker := tracker.New()
validWork := make(chan goimap.UID, 1)
tests := []struct {
name string
cfg Config
tracker *tracker.Tracker
work chan goimap.UID
wantErr string
}{
{
name: "nil tracker",
cfg: validConfig,
tracker: nil,
work: validWork,
wantErr: "missing tracker",
},
{
name: "nil work channel",
cfg: validConfig,
tracker: validTracker,
work: nil,
wantErr: "missing work channel",
},
{
name: "invalid config",
cfg: Config{
Mode: "invalid",
PollInterval: "30s",
},
tracker: validTracker,
work: validWork,
wantErr: "invalid config",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := NewWorker(
tt.cfg,
filter.Filters{},
imap.Config{},
nil, // log
tt.tracker,
tt.work,
)
if err == nil {
t.Fatal("NewWorker() expected error, got nil")
}
if !strings.Contains(err.Error(), tt.wantErr) {
t.Errorf(
"NewWorker() error = %q, want containing %q",
err.Error(), tt.wantErr,
)
}
})
}
}
+207
View File
@@ -0,0 +1,207 @@
// Package llm provides an OpenAI-compatible client for LLM interactions.
// It handles chat completions with automatic tool call execution.
package llm
import (
"context"
"fmt"
"log/slog"
"strings"
"time"
"github.com/chimerical-llc/raven/internal/message"
"github.com/chimerical-llc/raven/internal/tool"
openai "github.com/sashabaranov/go-openai"
)
// Config holds the configuration for an LLM client.
type Config struct {
// Key is the API key for authentication.
Key string `yaml:"key"`
// Model is the model identifier (e.g., "gpt-oss-120b", "qwen3-32b").
Model string `yaml:"model"`
// SystemPrompt is prepended to all conversations.
SystemPrompt string `yaml:"system_prompt"`
// Timeout is the maximum duration for a query (e.g., "15m").
// Defaults to 15 minutes if empty.
Timeout string `yaml:"timeout"`
// URL is the base URL of the OpenAI-compatible API endpoint.
URL string `yaml:"url"`
}
// Validate checks that required configuration values are present and valid.
func (cfg Config) Validate() error {
if cfg.Model == "" {
return fmt.Errorf("missing model")
}
if cfg.Timeout != "" {
if _, err := time.ParseDuration(cfg.Timeout); err != nil {
return fmt.Errorf("invalid duration")
}
}
if cfg.URL == "" {
return fmt.Errorf("missing URL")
}
return nil
}
// Client wraps an OpenAI-compatible client with tool execution support.
type Client struct {
client *openai.Client
log *slog.Logger
model string
registry *tool.Registry
systemPrompt string
timeout time.Duration
tools []openai.Tool
}
// NewClient creates a new LLM client with the provided configuration.
// The registry is optional; if nil, tool calling is disabled.
func NewClient(
cfg Config,
registry *tool.Registry,
log *slog.Logger,
) (*Client, error) {
if err := cfg.Validate(); err != nil {
return nil, fmt.Errorf("invalid config: %v", err)
}
var llm = &Client{
log: log,
model: cfg.Model,
systemPrompt: cfg.SystemPrompt,
registry: registry,
}
if cfg.Timeout == "" {
llm.timeout = time.Duration(15 * time.Minute)
} else {
d, err := time.ParseDuration(cfg.Timeout)
if err != nil {
return nil, fmt.Errorf("parse timeout: %v", err)
}
llm.timeout = d
}
// Setup client.
clientConfig := openai.DefaultConfig(cfg.Key)
clientConfig.BaseURL = cfg.URL
llm.client = openai.NewClientWithConfig(clientConfig)
// Parse tools.
if llm.registry != nil {
for _, name := range llm.registry.List() {
tool, _ := llm.registry.Get(name)
llm.tools = append(llm.tools, tool.OpenAI())
}
}
return llm, nil
}
// Query sends a message to the LLM and returns its response.
// It automatically executes any tool calls requested by the model,
// looping until the model returns a final text response.
// Supports multimodal content (text and images) via the Message type.
func (c *Client) Query(
ctx context.Context, msg *message.Message,
) (string, error) {
ctx, cancel := context.WithTimeout(ctx, c.timeout)
defer cancel()
// Build user message from email content.
// Uses MultiContent to support both text and image parts.
userMessage := openai.ChatCompletionMessage{
Role: openai.ChatMessageRoleUser,
MultiContent: msg.ToOpenAIMessages(),
}
// Set the system message and the user prompt.
messages := []openai.ChatCompletionMessage{
{
Role: openai.ChatMessageRoleSystem,
Content: c.systemPrompt,
},
userMessage,
}
// Loop for tool calls.
for {
req := openai.ChatCompletionRequest{
Model: c.model,
Messages: messages,
}
if len(c.tools) > 0 {
req.Tools = c.tools
}
res, err := c.client.CreateChatCompletion(ctx, req)
if err != nil {
return "", fmt.Errorf("chat completion: %w", err)
}
if len(res.Choices) == 0 {
return "", fmt.Errorf("no response choices returned")
}
choice := res.Choices[0]
message := choice.Message
// If no tool calls, we're done.
if len(message.ToolCalls) == 0 {
return message.Content, nil
}
// Add assistant message with tool calls to history.
messages = append(messages, message)
// Process each tool call.
for _, tc := range message.ToolCalls {
c.log.InfoContext(
ctx,
"calling tool",
slog.String("name", tc.Function.Name),
slog.String("args", tc.Function.Arguments),
)
res, err := c.registry.Execute(
ctx, tc.Function.Name, tc.Function.Arguments,
)
if err != nil {
// Return error to LLM so it can recover.
c.log.Error(
"failed to call tool",
slog.Any("error", err),
slog.String("name", tc.Function.Name),
)
// Assume JSON is a more helpful response
// for an LLM.
res = fmt.Sprintf(
`{"ok": false,"error": %q}`, err,
)
} else {
c.log.Info(
"called tool",
slog.String("name", tc.Function.Name),
)
}
// Content cannot be empty.
// Assume JSON is better than OK or (no output).
if strings.TrimSpace(res) == "" {
res = `{"ok":true,"result":null}`
}
// Add tool result to messages.
messages = append(
messages,
openai.ChatCompletionMessage{
Role: openai.ChatMessageRoleTool,
Content: res,
ToolCallID: tc.ID,
},
)
}
// Loop to get LLM's response after tool execution.
}
}
+91
View File
@@ -0,0 +1,91 @@
package llm
import (
"testing"
)
func TestConfigValidate(t *testing.T) {
tests := []struct {
name string
cfg Config
wantErr bool
}{
{
name: "empty config",
cfg: Config{},
wantErr: true,
},
{
name: "missing model",
cfg: Config{
URL: "https://api.example.com",
},
wantErr: true,
},
{
name: "missing URL",
cfg: Config{
Model: "gpt-4o",
},
wantErr: true,
},
{
name: "invalid timeout",
cfg: Config{
Model: "gpt-4o",
URL: "https://api.example.com",
Timeout: "not-a-duration",
},
wantErr: true,
},
{
name: "valid minimal config",
cfg: Config{
Model: "gpt-4o",
URL: "https://api.example.com",
},
wantErr: false,
},
{
name: "valid config with timeout",
cfg: Config{
Model: "gpt-4o",
URL: "https://api.example.com",
Timeout: "15m",
},
wantErr: false,
},
{
name: "valid config with complex timeout",
cfg: Config{
Model: "gpt-4o",
URL: "https://api.example.com",
Timeout: "1h30m45s",
},
wantErr: false,
},
{
name: "valid full config",
cfg: Config{
Key: "sk-test-key",
Model: "gpt-4o",
SystemPrompt: "You are a helpful assistant.",
Timeout: "30m",
URL: "https://api.example.com",
},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.cfg.Validate()
if (err != nil) != tt.wantErr {
t.Errorf(
"Validate() error = %v, wantErr %v",
err, tt.wantErr,
)
}
})
}
}
+529
View File
@@ -0,0 +1,529 @@
// Package message handles email message parsing and reply composition.
package message
import (
"bytes"
"encoding/base64"
"errors"
"fmt"
"io"
"log/slog"
"strings"
"time"
"github.com/emersion/go-imap/v2"
"github.com/emersion/go-imap/v2/imapclient"
_ "github.com/emersion/go-message/charset"
"github.com/emersion/go-message/mail"
openai "github.com/sashabaranov/go-openai"
)
// RFC5322 defines the date format specified in RFC 5322 §3.3.
const RFC5322 = "Mon, 2 Jan 2006 15:04:05 -0700"
// Supported image MIME types for vision models.
var supportedImageTypes = map[string]bool{
"image/png": true,
"image/jpeg": true,
"image/gif": true,
"image/webp": true,
}
// maxPartSize is the maximum size in bytes for a single MIME part (32MB).
const maxPartSize = 32 << 20
// Part represents a single piece of message content.
type Part struct {
// Content holds text content for text parts, empty for images.
Content string
// ContentType is the MIME type (e.g., "text/plain", "image/png").
ContentType string
// Data holds raw bytes for binary content like images.
Data []byte
// Filename is set for attachment parts.
Filename string
// IsAttachment distinguishes attachments from inline content.
IsAttachment bool
}
// IsImage returns true if the part is a supported image type.
func (p *Part) IsImage() bool {
return supportedImageTypes[p.ContentType]
}
// IsText returns true if the part contains text content.
func (p *Part) IsText() bool {
return strings.HasPrefix(p.ContentType, "text/")
}
// Message represents a parsed email with its metadata and content.
type Message struct {
// Attachments are parts with Content-Disposition: attachment.
// Stored separately to allow appending after inline content.
Attachments []Part
Envelope *imap.Envelope
// Parts contains inline content in order of appearance.
Parts []Part
// References are Message-IDs from the References header,
// used for threading.
References []string
UID imap.UID
log *slog.Logger
}
func (msg *Message) TextFrom() string {
var str strings.Builder
if msg.Envelope == nil || len(msg.Envelope.From) == 0 {
return ""
}
str.WriteString("From: ")
for i, a := range msg.Envelope.From {
if a.Name == "" {
fmt.Fprintf(&str, "%s", a.Addr())
} else {
fmt.Fprintf(
&str, "%s <%s>",
a.Name, a.Addr(),
)
}
if i+1 < len(msg.Envelope.From) {
str.WriteString(", ")
}
}
return str.String()
}
// TextBody returns the concatenated text content from all text parts.
// Used for reply composition and quoting.
func (msg *Message) TextBody() string {
var sb strings.Builder
first := true
for _, p := range msg.Parts {
if p.IsText() {
if !first {
sb.WriteString("\n")
}
sb.WriteString(p.Content)
first = false
}
}
return sb.String()
}
// ToOpenAIMessages converts the message content to OpenAI chat message parts.
// Inline parts appear first in order, followed by attachments.
// Text parts become text content, supported images become image_url content.
func (msg *Message) ToOpenAIMessages() []openai.ChatMessagePart {
var parts = []openai.ChatMessagePart{}
if msg != nil && msg.Envelope != nil {
var str strings.Builder
if v := msg.TextFrom(); v != "" {
fmt.Fprintf(&str, "%s\n", v)
}
if !msg.Envelope.Date.IsZero() {
fmt.Fprintf(
&str,
"Date: %s\n",
msg.Envelope.Date.Format(time.RFC3339),
)
}
if msg.Envelope.Subject != "" {
fmt.Fprintf(&str, "Subject: %v\n", msg.Envelope.Subject)
}
if v := str.String(); v != "" {
parts = append(
parts,
openai.ChatMessagePart{
Type: openai.ChatMessagePartTypeText,
Text: v,
},
)
}
}
// Process inline parts first, preserving order.
for _, p := range msg.Parts {
if part, ok := convertPart(p); ok {
parts = append(parts, part)
}
}
// Append attachments at the end.
for _, p := range msg.Attachments {
if part, ok := convertPart(p); ok {
parts = append(parts, part)
}
}
return parts
}
// convertPart converts a Part to an OpenAI ChatMessagePart.
// Returns false if the part type is not supported for LLM input.
func convertPart(p Part) (openai.ChatMessagePart, bool) {
switch {
case p.IsText():
return openai.ChatMessagePart{
Type: openai.ChatMessagePartTypeText,
Text: p.Content,
}, true
case p.IsImage():
dataURI := fmt.Sprintf(
"data:%s;base64,%s",
p.ContentType,
base64.StdEncoding.EncodeToString(p.Data),
)
return openai.ChatMessagePart{
Type: openai.ChatMessagePartTypeImageURL,
ImageURL: &openai.ChatMessageImageURL{
URL: dataURI,
Detail: openai.ImageURLDetailAuto,
},
}, true
default:
return openai.ChatMessagePart{}, false
}
}
// composeAttribution builds the attribution line for quoted replies.
// Returns sender and timestamp in a standard format like:
// "On Mon, 2 Jan 2006 15:04:05 -0700, raven <raven@example.com> wrote:"
func (msg *Message) composeAttribution() string {
if len(msg.Envelope.From) == 0 {
return "> \n"
}
from := msg.Envelope.From[0]
sender := from.Addr()
if from.Name != "" {
sender = fmt.Sprintf("%s <%s>", from.Name, from.Addr())
}
if msg.Envelope.Date.IsZero() {
return fmt.Sprintf("%s wrote:\n", sender)
}
return fmt.Sprintf(
"On %s, %s wrote:\n",
msg.Envelope.Date.Format(RFC5322),
sender,
)
}
// composeBody assembles the reply body: user's response, attribution line,
// and quoted original message.
func (msg *Message) composeBody(res string) string {
var body strings.Builder
body.WriteString(strings.TrimRight(res, "\n\r \t"))
body.WriteString("\n\n")
body.WriteString(msg.composeAttribution())
body.WriteString(msg.QuotedBody())
return body.String()
}
// composeHeader builds RFC 5322-compliant headers for a reply.
// Sets From, To, Subject, Date, Message-ID, In-Reply-To, and References.
func (msg *Message) composeHeader(
date time.Time, from *mail.Address,
) (*mail.Header, error) {
h := &mail.Header{}
h.SetDate(date)
h.SetContentType("text/plain", map[string]string{"charset": "utf-8"})
h.SetAddressList("From", []*mail.Address{from})
h.SetAddressList("Reply-To", []*mail.Address{from})
to := msg.composeRecipients()
if len(to) == 0 {
return nil, errors.New("missing recipients")
}
h.SetAddressList("To", to)
h.SetSubject(msg.composeSubject())
// Use sender's domain for Message-ID per RFC 5322 recommendation.
parts := strings.SplitN(from.Address, "@", 2)
if len(parts) == 2 {
if err := h.GenerateMessageIDWithHostname(parts[1]); err != nil {
return nil, fmt.Errorf("generate message id: %w", err)
}
} else {
if err := h.GenerateMessageID(); err != nil {
return nil, fmt.Errorf("generate message id: %w", err)
}
}
inReplyTo, refs := msg.composeReferences()
if inReplyTo != "" {
h.SetMsgIDList("In-Reply-To", []string{inReplyTo})
}
if len(refs) > 0 {
h.SetMsgIDList("References", refs)
}
return h, nil
}
// composeReferences builds threading headers per RFC 5322 §3.6.4.
// In-Reply-To contains the parent's Message-ID.
// References contains the full thread ancestry.
func (msg *Message) composeReferences() (inReplyTo string, refs []string) {
if msg.Envelope.MessageID != "" {
inReplyTo = msg.Envelope.MessageID
refs = append([]string(nil), msg.References...)
refs = append(refs, msg.Envelope.MessageID)
} else {
refs = append([]string(nil), msg.References...)
}
return
}
// composeRecipients determines the To address for a reply.
// Uses Reply-To if present, otherwise From (per RFC 5322 §3.6.2).
func (msg *Message) composeRecipients() []*mail.Address {
src := msg.Envelope.ReplyTo
if len(src) == 0 {
src = msg.Envelope.From
}
to := make([]*mail.Address, 0, len(src))
for _, v := range src {
to = append(to, &mail.Address{
Name: v.Name,
Address: v.Addr(),
})
}
return to
}
// composeSubject prepends "Re: " if not already present (case-insensitive).
func (msg *Message) composeSubject() string {
s := msg.Envelope.Subject
if !strings.HasPrefix(strings.ToLower(s), "re:") {
s = "Re: " + s
}
return s
}
// ComposeReply creates a reply to this message.
// The reply includes proper threading headers and the original message quoted.
func (msg *Message) ComposeReply(
date time.Time, from *mail.Address, res string,
) (*Reply, error) {
if msg == nil || msg.Envelope == nil {
return nil, errors.New("missing envelope")
}
if from == nil || from.Address == "" {
return nil, errors.New("missing from address")
}
header, err := msg.composeHeader(date, from)
if err != nil {
return nil, fmt.Errorf("compose header: %v", err)
}
return &Reply{body: msg.composeBody(res), header: header}, nil
}
// QuotedBody returns the message text body with each line prefixed by "> ".
func (msg *Message) QuotedBody() string {
var quoted strings.Builder
for line := range strings.SplitSeq(msg.TextBody(), "\n") {
line = strings.TrimSuffix(line, "\r")
quoted.WriteString("> ")
quoted.WriteString(line)
quoted.WriteString("\n")
}
return quoted.String()
}
// New creates a Message from an IMAP fetch buffer.
// Handles multipart messages by collecting text parts and images inline,
// with attachments stored separately for appending later.
// Logs skipped parts and non-fatal errors.
func New(mb *imapclient.FetchMessageBuffer, log *slog.Logger) (*Message, error) {
if mb == nil {
return nil, errors.New("nil message buffer")
}
if mb.UID == 0 {
return nil, errors.New("message has no UID")
}
if mb.Envelope == nil {
return nil, errors.New("message has no envelope")
}
if log == nil {
log = slog.Default()
}
msg := &Message{
UID: mb.UID,
Envelope: mb.Envelope,
log: log,
}
// Try each body section until we successfully parse one.
var parseErr error
for _, section := range mb.BodySection {
if len(section.Bytes) == 0 {
continue
}
if err := parseBody(
msg,
bytes.NewReader(section.Bytes),
log,
); err != nil {
parseErr = err
continue
}
if len(msg.Parts) > 0 {
break
}
}
// Failed to parse any content — return an error.
if len(msg.Parts) == 0 && parseErr != nil {
return msg, fmt.Errorf("parse body: %w", parseErr)
}
return msg, nil
}
// parseBody extracts content from a MIME message body.
// mail.Reader automatically flattens nested multipart structures, returning
// only leaf parts (text/plain, attachments, etc.).
func parseBody(msg *Message, r io.Reader, log *slog.Logger) error {
reader, err := mail.CreateReader(r)
if err != nil {
return fmt.Errorf("create reader: %w", err)
}
defer reader.Close()
// Extract References header for threading.
refs, err := reader.Header.MsgIDList("References")
if err == nil && len(refs) > 0 {
msg.References = refs
}
// Process all parts.
for {
part, err := reader.NextPart()
if errors.Is(err, io.EOF) {
break
}
if err != nil {
return fmt.Errorf("next part: %w", err)
}
if err := processPart(msg, part, log); err != nil {
log.Debug("skipped part",
slog.Any("error", err),
)
continue
}
}
return nil
}
// processPart handles a single MIME part returned by mail.Reader.
// Inline text and images are added to Parts; attachments go to Attachments.
func processPart(msg *Message, part *mail.Part, log *slog.Logger) error {
switch h := part.Header.(type) {
case *mail.InlineHeader:
ct, _, err := h.Header.ContentType()
if err != nil {
ct = "text/plain"
}
body, err := io.ReadAll(io.LimitReader(part.Body, maxPartSize))
if err != nil {
return fmt.Errorf("read body: %w", err)
}
switch {
case strings.HasPrefix(ct, "text/"):
msg.Parts = append(msg.Parts, Part{
Content: string(body),
ContentType: ct,
})
case supportedImageTypes[ct]:
msg.Parts = append(msg.Parts, Part{
ContentType: ct,
Data: body,
})
default:
log.Debug("skipped unsupported inline content type",
slog.String("content_type", ct),
)
}
case *mail.AttachmentHeader:
filename, _ := h.Filename()
ct, _, _ := h.Header.ContentType()
body, err := io.ReadAll(io.LimitReader(part.Body, maxPartSize))
if err != nil {
return fmt.Errorf("read attachment: %w", err)
}
// Only store attachments we can use (text or images).
switch {
case strings.HasPrefix(ct, "text/"):
msg.Attachments = append(msg.Attachments, Part{
Content: string(body),
ContentType: ct,
Filename: filename,
IsAttachment: true,
})
case supportedImageTypes[ct]:
msg.Attachments = append(msg.Attachments, Part{
ContentType: ct,
Data: body,
Filename: filename,
IsAttachment: true,
})
default:
log.Debug("skipped unsupported attachment type",
slog.String("content_type", ct),
slog.String("filename", filename),
)
}
}
return nil
}
// Reply holds a composed reply ready for sending.
type Reply struct {
body string
header *mail.Header
}
// Bytes serializes the reply to RFC 5322 wire format.
func (r *Reply) Bytes() ([]byte, error) {
var buf bytes.Buffer
mw, err := mail.CreateSingleInlineWriter(&buf, *r.header)
if err != nil {
return nil, fmt.Errorf("create writer: %w", err)
}
if _, err := mw.Write([]byte(r.body)); err != nil {
return nil, fmt.Errorf("write body: %w", err)
}
if err := mw.Close(); err != nil {
return nil, fmt.Errorf("close writer: %w", err)
}
return buf.Bytes(), nil
}
// Recipients returns the To addresses as formatted strings.
func (r *Reply) Recipients() ([]string, error) {
addrs, err := r.header.AddressList("To")
if err != nil {
return nil, fmt.Errorf("address list: %w", err)
}
to := make([]string, len(addrs))
for i, v := range addrs {
to[i] = v.Address
}
return to, nil
}
File diff suppressed because it is too large Load Diff
+219
View File
@@ -0,0 +1,219 @@
// Package service orchestrates the raven email processing pipeline.
// It coordinates intake workers (IMAP monitoring), answer workers (LLM
// processing and reply composition), and handles graceful shutdown.
package service
import (
"context"
"fmt"
"log/slog"
"os"
"os/signal"
"runtime/debug"
"strings"
"sync"
"syscall"
"time"
"github.com/chimerical-llc/raven/internal/answer"
"github.com/chimerical-llc/raven/internal/config"
"github.com/chimerical-llc/raven/internal/intake"
"github.com/chimerical-llc/raven/internal/llm"
"github.com/chimerical-llc/raven/internal/tool"
"github.com/chimerical-llc/raven/internal/tracker"
"github.com/emersion/go-imap/v2"
)
// Service is the main application coordinator.
// It owns the intake worker, answer workers, and shared resources like
// the LLM client and message tracker.
type Service struct {
cfg *config.Config
iw *intake.Worker
llm *llm.Client
log *slog.Logger
rw []*answer.Worker
tools *tool.Registry
tracker *tracker.Tracker
// work queues message UIDs for processing by answer workers.
work chan imap.UID
}
// Run starts the service and blocks until shutdown.
// It launches the intake worker to monitor IMAP for new messages and
// multiple answer workers to process them concurrently.
// Shutdown is triggered by SIGINT or SIGTERM.
func (svc *Service) Run(ctx context.Context) error {
svc.log.Info(
"starting raven",
slog.Int("concurrency", svc.cfg.Concurrency),
slog.Group(
"filters",
slog.String(
"body",
strings.Join(svc.cfg.Filters.Body, ","),
),
slog.String("from", svc.cfg.Filters.From),
slog.Any(
"senders",
strings.Join(svc.cfg.Filters.Senders, ","),
),
slog.String("subject", svc.cfg.Filters.Subject),
slog.String("to", svc.cfg.Filters.To),
),
slog.Group(
"imap",
slog.String("server", svc.cfg.IMAP.Address()),
slog.String("user", svc.cfg.IMAP.User),
),
slog.Group(
"intake",
slog.Any("mode", svc.cfg.Intake.Mode),
slog.Any("poll_interval", svc.cfg.Intake.PollInterval),
),
slog.Group(
"llm",
slog.String("url", svc.cfg.LLM.URL),
slog.String("model", svc.cfg.LLM.Model),
),
slog.Group(
"smtp",
slog.String("server", svc.cfg.SMTP.Address()),
slog.String("user", svc.cfg.SMTP.User),
),
slog.Any("shutdown_timeout", svc.cfg.GetShutdownTimeout()),
slog.Group(
"tools",
slog.Int("count", len(svc.tools.List())),
slog.Any(
"names",
strings.Join(svc.tools.List(), ","),
),
),
)
if len(svc.cfg.Filters.Senders) == 0 {
svc.log.Warn("ACCEPTING MAIL FROM ANY SENDER")
}
// Coordinate worker shutdown.
var wg sync.WaitGroup
// Setup signal handling for graceful shutdown.
ctx, cancel := signal.NotifyContext(
ctx,
os.Interrupt, syscall.SIGTERM,
)
defer cancel()
// Start intake worker.
wg.Go(func() {
defer func() {
if r := recover(); r != nil {
svc.log.Error(
"intake worker panicked",
slog.Any("panic", r),
slog.String(
"stack", string(debug.Stack()),
),
)
}
close(svc.work)
cancel()
}()
if err := svc.iw.Run(ctx); err != nil {
svc.log.ErrorContext(
ctx,
"failed to run intake worker",
slog.Any("error", err),
)
}
})
// Start processor workers.
for _, p := range svc.rw {
wg.Go(func() { p.Run(ctx) })
}
// Wait for shutdown signal.
<-ctx.Done()
svc.log.Info("shutdown signal received")
// Wait for graceful shutdown with timeout.
done := make(chan struct{})
go func() {
wg.Wait()
close(done)
}()
select {
case <-done:
svc.log.Info("all workers stopped")
svc.log.Info("terminating")
case <-time.After(svc.cfg.GetShutdownTimeout()):
svc.log.Warn("shutdown timeout reached; workers may still be running")
svc.log.Info("terminating")
}
return nil
}
// New creates a Service from the provided configuration.
// It initializes the tool registry, LLM client, intake worker,
// and the configured number of answer workers.
func New(cfg *config.Config, log *slog.Logger) (*Service, error) {
var svc = &Service{
cfg: cfg,
log: log,
tracker: tracker.New(),
work: make(chan imap.UID),
}
// Setup clients and tools.
registry, err := tool.NewRegistry(cfg.Tools)
if err != nil {
return nil, fmt.Errorf("load tools: %v", err)
}
svc.tools = registry
// Create the LLM client.
llmClient, err := llm.NewClient(cfg.LLM, registry, log)
if err != nil {
return nil, fmt.Errorf("create LLM client: %v", err)
}
svc.llm = llmClient
// Create the IMAP worker.
w, err := intake.NewWorker(
svc.cfg.Intake,
svc.cfg.Filters,
svc.cfg.IMAP,
svc.log,
svc.tracker,
svc.work,
)
if err != nil {
return nil, fmt.Errorf("create intake worker: %v", err)
}
svc.iw = w
// Create processors.
for id := range svc.cfg.Concurrency {
w, err := answer.NewWorker(
id,
svc.cfg.Filters,
svc.cfg.IMAP,
svc.cfg.SMTP,
svc.llm,
svc.tracker,
svc.log,
svc.work,
)
if err != nil {
return nil, fmt.Errorf("create answer worker: %v", err)
}
svc.rw = append(svc.rw, w)
}
return svc, nil
}
+66
View File
@@ -0,0 +1,66 @@
// Package smtp provides a config wrapper around net/smtp for sending email.
package smtp
import (
"fmt"
"net/mail"
"net/smtp"
)
// SMTP holds configuration for sending emails via an SMTP server.
type SMTP struct {
From string `yaml:"from"`
Host string `yaml:"host"`
Password string `yaml:"password"`
Port string `yaml:"port"`
User string `yaml:"user"`
}
// Address returns the host:port string.
func (s *SMTP) Address() string {
return fmt.Sprintf("%s:%s", s.Host, s.Port)
}
// Auth creates a PlainAuth using the configured credentials.
func (s *SMTP) Auth() smtp.Auth {
return smtp.PlainAuth("", s.User, s.Password, s.Host)
}
// Send emails the given recipients using net/smtp.
// Refer to smtp.SendMail for its parameters and limitations.
func (s *SMTP) Send(to []string, msg []byte) error {
if err := smtp.SendMail(
s.Address(),
s.Auth(),
s.From,
to,
msg,
); err != nil {
return fmt.Errorf("smtp send: %w", err)
}
return nil
}
// Validate checks that all required configuration fields are set.
func (s *SMTP) Validate() error {
if s.From == "" {
return fmt.Errorf("missing from")
}
if _, err := mail.ParseAddress(s.From); err != nil {
return fmt.Errorf("invalid from: %w", err)
}
if s.Host == "" {
return fmt.Errorf("missing host")
}
if s.Password == "" {
return fmt.Errorf("missing password")
}
if s.Port == "" {
return fmt.Errorf("missing port")
}
if s.User == "" {
return fmt.Errorf("missing user")
}
return nil
}
+111
View File
@@ -0,0 +1,111 @@
package smtp
import (
"testing"
)
func TestAddress(t *testing.T) {
s := &SMTP{
Host: "smtp.example.com",
Port: "587",
}
expected := "smtp.example.com:587"
if got := s.Address(); got != expected {
t.Errorf("Address() = %q, want %q", got, expected)
}
}
func TestAuth(t *testing.T) {
s := &SMTP{
User: "user",
Password: "password",
Host: "smtp.example.com",
}
// Verify return of non-nil Auth mechanism.
if got := s.Auth(); got == nil {
t.Error("Auth() returned nil")
}
}
func TestValidate(t *testing.T) {
tests := []struct {
name string
s *SMTP
shouldError bool
}{
{
name: "valid configuration",
s: &SMTP{
From: "sender@example.com",
Host: "smtp.example.com",
Password: "secretpassword",
Port: "587",
User: "user",
},
shouldError: false,
},
{
name: "missing from",
s: &SMTP{
Host: "smtp.example.com",
Password: "p",
Port: "587",
User: "u",
},
shouldError: true,
},
{
name: "missing host",
s: &SMTP{
From: "f",
Password: "p",
Port: "587",
User: "u",
},
shouldError: true,
},
{
name: "missing password",
s: &SMTP{
From: "f",
Host: "h",
Port: "587",
User: "u",
},
shouldError: true,
},
{
name: "missing port",
s: &SMTP{
From: "f",
Host: "h",
Password: "p",
User: "u",
},
shouldError: true,
},
{
name: "missing user",
s: &SMTP{
From: "f",
Host: "h",
Password: "p",
Port: "587",
},
shouldError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.s.Validate()
if tt.shouldError && err == nil {
t.Fatalf("expected error, got nil")
}
if !tt.shouldError && err != nil {
t.Fatalf("unexpected error: %v", err)
}
})
}
}
+235
View File
@@ -0,0 +1,235 @@
// Package tool provides a registry for external tools that can be invoked by
// LLMs.
//
// The package bridges YAML configuration to exec.CommandContext, allowing
// tools to be defined declaratively without writing Go code. Each tool
// specifies a command, argument templates using Go's text/template syntax,
// JSON Schema parameters for LLM input, and an optional execution timeout.
//
// The registry validates all tool definitions at construction time,
// failing fast on configuration errors. At execution time, it expands
// argument templates with LLM-provided JSON, runs the subprocess, and
// returns stdout.
package tool
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os/exec"
"text/template"
"time"
"github.com/sashabaranov/go-openai"
)
// fn provides template functions available to argument templates.
var fn = template.FuncMap{
"json": func(v any) string {
b, _ := json.Marshal(v)
return string(b)
},
}
// Tool represents an external tool that can be invoked by LLMs.
// Tools are executed as subprocesses with templated arguments.
type Tool struct {
// Name uniquely identifies the tool within a registry.
Name string `yaml:"name"`
// Description explains the tool's purpose for the LLM.
Description string `yaml:"description"`
// Command is the executable path or name.
Command string `yaml:"command"`
// Arguments are Go templates expanded with LLM-provided parameters.
// Empty results after expansion are filtered out.
Arguments []string `yaml:"arguments"`
// Parameters is a JSON Schema describing expected input from the LLM.
Parameters map[string]any `yaml:"parameters"`
// Timeout limits execution time (e.g., "30s", "5m").
// Empty means no timeout.
Timeout string `yaml:"timeout"`
// timeout is the parsed duration, set during registry construction.
timeout time.Duration `yaml:"-"`
}
// OpenAI converts the tool to an OpenAI function definition for API calls.
func (t Tool) OpenAI() openai.Tool {
return openai.Tool{
Type: openai.ToolTypeFunction,
Function: &openai.FunctionDefinition{
Name: t.Name,
Description: t.Description,
Parameters: t.Parameters,
},
}
}
// ParseArguments expands argument templates with the provided JSON data.
// The args parameter should be a JSON object string; empty string or "{}"
// results in an empty data map. Templates producing empty strings are
// filtered from the result, allowing conditional arguments.
func (t Tool) ParseArguments(args string) ([]string, error) {
var data = map[string]any{}
if args != "" && args != "{}" {
if err := json.Unmarshal([]byte(args), &data); err != nil {
return nil, fmt.Errorf(
"invalid arguments JSON: %w", err,
)
}
}
var result []string
for _, v := range t.Arguments {
tmpl, err := template.New("").Funcs(fn).Parse(v)
if err != nil {
return nil, fmt.Errorf(
"invalid template %q: %w", v, err,
)
}
var buf bytes.Buffer
if err := tmpl.Execute(&buf, data); err != nil {
return nil, fmt.Errorf(
"execute template %q: %w", v, err,
)
}
// Filter out empty strings (unused conditional arguments).
if s := buf.String(); s != "" {
result = append(result, s)
}
}
return result, nil
}
// Validate checks that the tool definition is complete and valid.
// It verifies required fields are present, the timeout (if specified)
// is parseable, and all argument templates are syntactically valid.
func (t Tool) Validate() error {
if t.Name == "" {
return fmt.Errorf("missing name")
}
if t.Description == "" {
return fmt.Errorf("missing description")
}
if t.Command == "" {
return fmt.Errorf("missing command")
}
if t.Timeout != "" {
if _, err := time.ParseDuration(t.Timeout); err != nil {
return fmt.Errorf("invalid timeout: %v", err)
}
}
for _, arg := range t.Arguments {
if _, err := template.New("").Funcs(fn).Parse(arg); err != nil {
return fmt.Errorf("invalid argument template")
}
}
return nil
}
// Registry holds tools indexed by name and handles their execution.
// It validates all tools at construction time to fail fast on
// configuration errors.
type Registry struct {
tools map[string]*Tool
}
// NewRegistry creates a registry from the provided tool definitions.
// Returns an error if any tool fails validation or if duplicate names exist.
func NewRegistry(tools []Tool) (*Registry, error) {
var r = &Registry{
tools: make(map[string]*Tool),
}
for _, t := range tools {
if err := t.Validate(); err != nil {
return nil, fmt.Errorf("invalid tool: %v", err)
}
if t.Timeout != "" {
d, err := time.ParseDuration(t.Timeout)
if err != nil {
return nil, fmt.Errorf(
"parse timeout: %v", err,
)
}
t.timeout = d
}
if _, exists := r.tools[t.Name]; exists {
return nil, fmt.Errorf(
"duplicate tool name: %s", t.Name,
)
}
r.tools[t.Name] = &t
}
return r, nil
}
// Get returns a tool by name and a boolean indicating if it was found.
func (r *Registry) Get(name string) (*Tool, bool) {
tool, ok := r.tools[name]
return tool, ok
}
// List returns all registered tool names in arbitrary order.
func (r *Registry) List() []string {
names := make([]string, 0, len(r.tools))
for name := range r.tools {
names = append(names, name)
}
return names
}
// Execute runs a tool by name with the provided JSON arguments.
// It expands argument templates, executes the command as a subprocess,
// and returns stdout on success. The context can be used for cancellation;
// tool-specific timeouts are applied on top of any context deadline.
func (r *Registry) Execute(
ctx context.Context, name string, args string,
) (string, error) {
tool, ok := r.tools[name]
if !ok {
return "", fmt.Errorf("unknown tool: %s", name)
}
// Evaluate argument templates.
cmdArgs, err := tool.ParseArguments(args)
if err != nil {
return "", fmt.Errorf("parse arguments: %w", err)
}
// If defined, use the timeout.
if tool.timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, tool.timeout)
defer cancel()
}
// Setup and run the command.
var (
stdout, stderr bytes.Buffer
cmd = exec.CommandContext(
ctx, tool.Command, cmdArgs...,
)
)
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
if ctx.Err() == context.DeadlineExceeded && tool.timeout > 0 {
return "", fmt.Errorf(
"tool %s timed out after %v",
name, tool.timeout,
)
}
return "", fmt.Errorf(
"tool %s: %w\nstderr: %s",
name, err, stderr.String(),
)
}
return stdout.String(), nil
}
+374
View File
@@ -0,0 +1,374 @@
package tool
import (
"context"
"slices"
"testing"
"time"
)
func TestToolValidate(t *testing.T) {
tests := []struct {
name string
tool Tool
wantErr bool
}{
{
name: "valid tool",
tool: Tool{
Name: "echo",
Description: "echoes input",
Command: "echo",
Arguments: []string{"{{.message}}"},
},
wantErr: false,
},
{
name: "valid tool with timeout",
tool: Tool{
Name: "slow",
Description: "slow command",
Command: "sleep",
Arguments: []string{"1"},
Timeout: "5s",
},
wantErr: false,
},
{
name: "missing name",
tool: Tool{
Description: "test",
Command: "echo",
},
wantErr: true,
},
{
name: "missing description",
tool: Tool{
Name: "test",
Command: "echo",
},
wantErr: true,
},
{
name: "missing command",
tool: Tool{
Name: "test",
Description: "test",
},
wantErr: true,
},
{
name: "invalid timeout",
tool: Tool{
Name: "test",
Description: "test",
Command: "echo",
Timeout: "invalid",
},
wantErr: true,
},
{
name: "invalid template",
tool: Tool{
Name: "test",
Description: "test",
Command: "echo",
Arguments: []string{"{{.unclosed"},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.tool.Validate()
if (err != nil) != tt.wantErr {
t.Errorf("Validate() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestToolParseArguments(t *testing.T) {
tests := []struct {
name string
tool Tool
args string
want []string
wantErr bool
}{
{
name: "simple substitution",
tool: Tool{
Arguments: []string{"{{.message}}"},
},
args: `{"message": "hello"}`,
want: []string{"hello"},
},
{
name: "multiple arguments",
tool: Tool{
Arguments: []string{"-n", "{{.count}}", "{{.file}}"},
},
args: `{"count": "10", "file": "test.txt"}`,
want: []string{"-n", "10", "test.txt"},
},
{
name: "conditional with if",
tool: Tool{
Arguments: []string{"{{.required}}", "{{if .optional}}{{.optional}}{{end}}"},
},
args: `{"required": "value"}`,
want: []string{"value"},
},
{
name: "json function",
tool: Tool{
Arguments: []string{`{{json .data}}`},
},
args: `{"data": {"key": "value"}}`,
want: []string{`{"key":"value"}`},
},
{
name: "empty JSON object",
tool: Tool{
Arguments: []string{"fixed"},
},
args: "{}",
want: []string{"fixed"},
},
{
name: "empty string args",
tool: Tool{
Arguments: []string{"fixed"},
},
args: "",
want: []string{"fixed"},
},
{
name: "invalid JSON",
tool: Tool{
Arguments: []string{"{{.x}}"},
},
args: "not json",
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := tt.tool.ParseArguments(tt.args)
if (err != nil) != tt.wantErr {
t.Errorf("ParseArguments() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !tt.wantErr && !slices.Equal(got, tt.want) {
t.Errorf("ParseArguments() = %v, want %v", got, tt.want)
}
})
}
}
func TestNewRegistry(t *testing.T) {
tests := []struct {
name string
tools []Tool
wantErr bool
}{
{
name: "valid registry",
tools: []Tool{
{Name: "a", Description: "a", Command: "echo"},
{Name: "b", Description: "b", Command: "cat"},
},
wantErr: false,
},
{
name: "empty registry",
tools: []Tool{},
wantErr: false,
},
{
name: "duplicate names",
tools: []Tool{
{Name: "dup", Description: "first", Command: "echo"},
{Name: "dup", Description: "second", Command: "cat"},
},
wantErr: true,
},
{
name: "invalid tool",
tools: []Tool{
{Name: "", Description: "missing name", Command: "echo"},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := NewRegistry(tt.tools)
if (err != nil) != tt.wantErr {
t.Errorf("NewRegistry() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestRegistryGet(t *testing.T) {
r, err := NewRegistry([]Tool{
{Name: "echo", Description: "echo", Command: "echo"},
})
if err != nil {
t.Fatalf("NewRegistry() error = %v", err)
}
tool, ok := r.Get("echo")
if !ok {
t.Error("Get(echo) returned false, want true")
}
if tool.Name != "echo" {
t.Errorf("Get(echo).Name = %q, want %q", tool.Name, "echo")
}
_, ok = r.Get("nonexistent")
if ok {
t.Error("Get(nonexistent) returned true, want false")
}
}
func TestRegistryList(t *testing.T) {
r, err := NewRegistry([]Tool{
{Name: "a", Description: "a", Command: "echo"},
{Name: "b", Description: "b", Command: "cat"},
{Name: "c", Description: "c", Command: "ls"},
})
if err != nil {
t.Fatalf("NewRegistry() error = %v", err)
}
names := r.List()
if len(names) != 3 {
t.Errorf("List() returned %d names, want 3", len(names))
}
slices.Sort(names)
want := []string{"a", "b", "c"}
if !slices.Equal(names, want) {
t.Errorf("List() = %v, want %v", names, want)
}
}
func TestRegistryExecute(t *testing.T) {
r, err := NewRegistry([]Tool{
{
Name: "echo",
Description: "echo message",
Command: "echo",
Arguments: []string{"{{.message}}"},
},
{
Name: "cat",
Description: "cat stdin",
Command: "cat",
Arguments: []string{},
},
})
if err != nil {
t.Fatalf("NewRegistry() error = %v", err)
}
ctx := context.Background()
// Successful execution.
out, err := r.Execute(ctx, "echo", `{"message": "hello world"}`)
if err != nil {
t.Errorf("Execute(echo) error = %v", err)
}
if out != "hello world\n" {
t.Errorf("Execute(echo) = %q, want %q", out, "hello world\n")
}
// Unknown tool.
_, err = r.Execute(ctx, "unknown", "{}")
if err == nil {
t.Error("Execute(unknown) expected error, got nil")
}
// Invalid JSON arguments.
_, err = r.Execute(ctx, "echo", "not json")
if err == nil {
t.Error("Execute with invalid JSON expected error, got nil")
}
}
func TestRegistryExecuteTimeout(t *testing.T) {
r, err := NewRegistry([]Tool{
{
Name: "slow",
Description: "slow command",
Command: "sleep",
Arguments: []string{"10"},
Timeout: "50ms",
},
})
if err != nil {
t.Fatalf("NewRegistry() error = %v", err)
}
ctx := context.Background()
start := time.Now()
_, err = r.Execute(ctx, "slow", "{}")
elapsed := time.Since(start)
if err == nil {
t.Error("Execute(slow) expected timeout error, got nil")
}
if elapsed > time.Second {
t.Errorf("Execute(slow) took %v, expected ~50ms timeout", elapsed)
}
}
func TestRegistryExecuteFailure(t *testing.T) {
r, err := NewRegistry([]Tool{
{
Name: "fail",
Description: "always fails",
Command: "false",
},
})
if err != nil {
t.Fatalf("NewRegistry() error = %v", err)
}
_, err = r.Execute(context.Background(), "fail", "{}")
if err == nil {
t.Error("Execute(fail) expected error, got nil")
}
}
func TestToolOpenAI(t *testing.T) {
tool := Tool{
Name: "test",
Description: "test tool",
Command: "echo",
Parameters: map[string]any{
"type": "object",
"properties": map[string]any{
"message": map[string]any{
"type": "string",
"description": "the message",
},
},
},
}
openaiTool := tool.OpenAI()
if openaiTool.Function.Name != "test" {
t.Errorf("OpenAI().Function.Name = %q, want %q", openaiTool.Function.Name, "test")
}
if openaiTool.Function.Description != "test tool" {
t.Errorf("OpenAI().Function.Description = %q, want %q", openaiTool.Function.Description, "test tool")
}
}
+43
View File
@@ -0,0 +1,43 @@
// Package tracker prevents duplicate processing of IMAP messages by
// maintaining a transient set of active, in-flight UIDs.
package tracker
import (
"sync"
"github.com/emersion/go-imap/v2"
)
// Tracker ensures unique concurrent processing of messages.
// This prevents race conditions where a message might be fetched
// again before its initial processing completes.
type Tracker struct {
mu sync.Mutex
ids map[imap.UID]struct{}
}
// New returns a ready-to-use Tracker.
func New() *Tracker {
return &Tracker{
ids: make(map[imap.UID]struct{}),
}
}
// TryAcquire attempts to claim exclusive processing rights for a UID.
// It returns true only if the UID is not currently being tracked.
func (t *Tracker) TryAcquire(uid imap.UID) bool {
t.mu.Lock()
defer t.mu.Unlock()
if _, exists := t.ids[uid]; exists {
return false
}
t.ids[uid] = struct{}{}
return true
}
// Release relinquishes processing rights for a UID, allowing it to be acquired again.
func (t *Tracker) Release(uid imap.UID) {
t.mu.Lock()
defer t.mu.Unlock()
delete(t.ids, uid)
}
+94
View File
@@ -0,0 +1,94 @@
package tracker
import (
"sync"
"testing"
"github.com/emersion/go-imap/v2"
)
func TestNew(t *testing.T) {
tr := New()
if tr == nil {
t.Fatal("New() returned nil")
}
if tr.ids == nil {
t.Fatal("New() initialized with nil map")
}
}
func TestLifecycle(t *testing.T) {
tr := New()
uid := imap.UID(123)
// 1. First acquire should succeed.
if !tr.TryAcquire(uid) {
t.Fatalf(
"expected TryAcquire(%d) to return true, got false",
uid,
)
}
// 2. Second acquire on same UID should fail.
if tr.TryAcquire(uid) {
t.Fatalf(
"expected TryAcquire(%d) to return false when already held, got true",
uid,
)
}
// 3. Different UID should succeed.
otherUID := imap.UID(456)
if !tr.TryAcquire(otherUID) {
t.Fatalf(
"expected TryAcquire(%d) to return true, got false",
otherUID,
)
}
// 4. Release first UID.
tr.Release(uid)
// 5. First UID should be acquirable again.
if !tr.TryAcquire(uid) {
t.Fatalf(
"expected TryAcquire(%d) to return true after Release, got false",
uid,
)
}
}
func TestConcurrency(t *testing.T) {
tr := New()
uid := imap.UID(1)
routines := 16
var wg sync.WaitGroup
// Counter for successful acquisitions.
successCount := 0
var mu sync.Mutex
for range routines {
wg.Go(func() {
if tr.TryAcquire(uid) {
mu.Lock()
successCount++
mu.Unlock()
}
})
}
wg.Wait()
if successCount != 1 {
t.Errorf(
"expected exactly 1 successful acquisition for concurrent access to same UID, got %d",
successCount,
)
}
// Verify we can release safely after concurrent hammer.
tr.Release(uid)
if !tr.TryAcquire(uid) {
t.Error("failed to acquire UID after concurrent test release")
}
}