add project README
This commit is contained in:
parent
992381f8d9
commit
f7a11fa595
|
|
@ -28,5 +28,5 @@ dist/
|
|||
build/
|
||||
*.egg
|
||||
*.egg-info
|
||||
|
||||
output/
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,146 @@
|
|||
# voice-pipeline
|
||||
|
||||
`voice-pipeline` is a three-process Python service for collecting call recordings, analyzing them, and sending results back to Bitrix24.
|
||||
|
||||
## What it does
|
||||
|
||||
- `collector` fetches call metadata and recordings from Bitrix24 / Asterisk sources and writes them into a local spool directory.
|
||||
- `analyzer` picks up spooled calls, transcribes them, performs diarization and script compliance checks, and stores artifacts next to the call.
|
||||
- `reporter` creates Bitrix24 activities and HTML reports from analyzed calls.
|
||||
|
||||
The processes communicate through the spool directory and the shared `meta.json` contract stored inside each call folder.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python `3.11+`
|
||||
- Linux for the systemd deployment
|
||||
- A configured Bitrix24 webhook
|
||||
- Optional dependencies depending on your use case:
|
||||
- `paramiko` for Asterisk SFTP collection
|
||||
- `pyannote.audio` for mixed-audio diarization
|
||||
- `openai`, `groq`, `transformers`, `torch`, or `faster-whisper` for the analyzer, depending on backend choice
|
||||
|
||||
## Installation
|
||||
|
||||
Create a virtual environment and install the project in editable mode:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
Install optional extras when needed:
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev,diarize]" # mixed-audio diarization via pyannote
|
||||
pip install -e ".[dev,local-asr]" # local Whisper-compatible transcription
|
||||
pip install -e ".[dev,emotion]" # local emotion classification
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Copy the example config and edit it:
|
||||
|
||||
```bash
|
||||
cp config/config.example.toml /etc/voice-pipeline/config.toml
|
||||
```
|
||||
|
||||
The config path is read from `VOICE_PIPELINE_CONFIG`, or defaults to:
|
||||
|
||||
```text
|
||||
/etc/voice-pipeline/config.toml
|
||||
```
|
||||
|
||||
Important sections:
|
||||
|
||||
- `[storage]` controls spool, reports, and database paths.
|
||||
- `[[sources.asterisk]]` defines one or more Asterisk SFTP/HTTP sources.
|
||||
- `[sources.bitrix24]` defines the Bitrix24 webhook and collector options.
|
||||
- `[analyzer]` configures transcription, diarization, emotions, and script files.
|
||||
- `[reporter]` controls report size and summary window.
|
||||
|
||||
Run a validation check after editing the config:
|
||||
|
||||
```bash
|
||||
voice-pipeline validate-config
|
||||
```
|
||||
|
||||
## CLI
|
||||
|
||||
The project exposes a single command:
|
||||
|
||||
```bash
|
||||
voice-pipeline validate-config
|
||||
voice-pipeline collector
|
||||
voice-pipeline analyzer
|
||||
voice-pipeline reporter
|
||||
voice-pipeline reload --list --date 2026-06-23
|
||||
voice-pipeline reload --reanalyze --uniqueid 1782211489.118028
|
||||
voice-pipeline reload --recollect --status failed
|
||||
```
|
||||
|
||||
`reload` supports three modes:
|
||||
|
||||
- `--list` shows matching calls without changing anything.
|
||||
- `--reanalyze` resets selected calls back to `spooled` so analyzer can run again.
|
||||
- `--recollect` deletes the selected calls from DB and spool so the collector can fetch them again.
|
||||
|
||||
## Collector behavior
|
||||
|
||||
The collector currently uses `voximplant.statistic.get` as the main source of truth.
|
||||
|
||||
- Calls are filtered by day and by minimum `CALL_DURATION`.
|
||||
- The collector enriches each call with CRM entity and Bitrix user data.
|
||||
- Recording download depends on `REST_APP_NAME`:
|
||||
- "My Calls" calls are downloaded via `CALL_RECORD_URL`.
|
||||
- `Asterisk` calls are resolved in the Asterisk storage by `EXTERNAL_CALL_ID`.
|
||||
- Partial Asterisk matches can be spooled when only some of `mixed/rx/tx` are present.
|
||||
|
||||
Each spooled call gets a `meta.json` file with the structured metadata used by the analyzer and reporter.
|
||||
|
||||
## Analyzer behavior
|
||||
|
||||
- Asterisk split-channel calls use `rx.wav` and `tx.wav`.
|
||||
- Mixed audio calls use `audio.mp3` or fallback `mixed.wav` when available.
|
||||
- Transcript and analysis artifacts are written next to the call folder.
|
||||
|
||||
## Systemd deployment
|
||||
|
||||
The repository includes unit files and an installer script:
|
||||
|
||||
```bash
|
||||
sudo bash systemd/install.sh
|
||||
```
|
||||
|
||||
The script creates the `voice-pipeline` system user, installs the unit files, and prepares the spool and reports directories under `/var/lib/voice-pipeline`.
|
||||
|
||||
## Tests
|
||||
|
||||
Run the test suite with:
|
||||
|
||||
```bash
|
||||
pytest -q
|
||||
```
|
||||
|
||||
For targeted runs:
|
||||
|
||||
```bash
|
||||
pytest tests/test_collector.py -q
|
||||
pytest tests/test_integration.py -q
|
||||
```
|
||||
|
||||
## Repository layout
|
||||
|
||||
- `src/collector/` - collection pipeline, Bitrix24/Asterisk helpers, spooler
|
||||
- `src/analyzer/` - transcription, diarization, emotion scoring, script checks
|
||||
- `src/reporter/` - Bitrix24 push and HTML report generation
|
||||
- `src/core/` - shared config, database, models, logging, retries
|
||||
- `systemd/` - service units and install script
|
||||
- `config/` - example configuration
|
||||
- `tests/` - unit and integration tests
|
||||
|
||||
## Notes
|
||||
|
||||
- `docs/README.md` contains additional operator notes for the reload workflow.
|
||||
- `meta.json` is written last and acts as the readiness signal for downstream processes.
|
||||
Loading…
Reference in New Issue