GitHub

Development Documentation

This comprehensive guide covers Planar's architecture, development setup, contribution guidelines, and best practices for extending the framework.

Architecture Overview

System Architecture

Planar follows a modular architecture with clear separation of concerns:

┌─────────────────────────────────────────────────────────────────────────────┐
│                              Application Layer                              │
├─────────────────────────────────────────────────────────────────────────────┤
│  Planar.jl (Engine, LiveMode, PaperMode,   │  PlanarOptim.jl                │
├─────────────────────────────────────────────────────────────────────────────┤
│   Watchers, Remote)                        │  (Plotting)                    │
├─────────────────────────────────────────────────────────────────────────────┤
│                                Strategy Layer                               │
├─────────────────────────────────────────────────────────────────────────────┤
│  Strategies.jl    │   User Strategies    │ PlanarStrategyStats/             │
├─────────────────────────────────────────────────────────────────────────────┤
│                   │                      │ PlanarStrategyTools              │
├─────────────────────────────────────────────────────────────────────────────┤
│                               Foundation Layer                              │
├─────────────────────────────────────────────────────────────────────────────┤
│  PlanarCore (Lang, TimeTicks, Pbar, Misc, Ccxt,                             │
├─────────────────────────────────────────────────────────────────────────────┤
│   ExchangeTypes, Data, Instruments, Processing, Exchanges,                  │
├─────────────────────────────────────────────────────────────────────────────┤
│   Fetch, OrderTypes, Instances, Collections, Strategies,                    │
├─────────────────────────────────────────────────────────────────────────────┤
│   Simulations, Executors, SimMode, Metrics, Stubs)                          │
├─────────────────────────────────────────────────────────────────────────────┤
│                                Tooling Layer                                │
├─────────────────────────────────────────────────────────────────────────────┤
│  PlanarDownloadTool  │  PlanarPython  │  PlanarFeatureSelection │  PlanarDev│
└─────────────────────────────────────────────────────────────────────────────┘

Core Components

Engine Module

The foundation of Planar, providing:

  • Asset Management: AbstractAsset, Asset, Derivative types
  • Instance System: AssetInstance linking assets to exchanges
  • Strategy Framework: Strategy{Mode} parametric types
  • Core Abstractions: Base types and interfaces

Execution Modes

Three distinct execution environments:

Data Pipeline

Comprehensive data management:

  • Fetch: Data acquisition from exchanges and external sources
  • Processing: Data cleaning, resampling, and transformation
  • Storage: Zarr-based large dataset storage with LMDB indexing
  • Watchers: Real-time data monitoring and alerting

Exchange Integration

Unified exchange interface:

  • CCXT Integration: 100+ exchange support via Ccxt.jl
  • Custom Exchanges: Framework for implementing proprietary APIs
  • Order Management: Unified order types across exchanges

Module Relationships

graph TD
    subgraph PL["Planar (application layer)"]
        O[Planar.jl]
        E[Engine]
        PM[PaperMode]
        LM[LiveMode]
        WA[Watchers]
        RE[Remote]
        O --> E
        O --> PM
        O --> LM
        O --> WA
        O --> RE
    end

    subgraph PC["PlanarCore (foundation layer)"]
        L[Lang]
        TT[TimeTicks]
        PB[Pbar]
        MI[Misc]
        CX[Ccxt]
        ET[ExchangeTypes]
        DA[Data]
        IN[Instruments]
        PR[Processing]
        EX[Exchanges]
        FE[Fetch]
        OT[OrderTypes]
        IS[Instances]
        CO[Collections]
        ST[Strategies]
        SI[Simulations]
        EC[Executors]
        SM[SimMode]
        ME[Metrics]
        SB[Stubs]

        TT --> L
        PB --> L
        PB --> TT
        MI --> TT
        CX --> MI
        ET --> CX
        DA --> L
        DA --> MI
        DA --> TT
        IN --> L
        IN --> MI
        PR --> DA
        PR --> MI
        PR --> PB
        EX --> CX
        EX --> DA
        EX --> ET
        EX --> IN
        EX --> PB
        FE --> CX
        FE --> DA
        FE --> EX
        FE --> IN
        FE --> MI
        FE --> PR
        FE --> TT
        OT --> ET
        OT --> IN
        IS --> EX
        IS --> OT
        CO --> DA
        CO --> IS
        CO --> IN
        CO --> L
        CO --> MI
        CO --> TT
        ST --> CO
        ST --> DA
        ST --> IS
        ST --> IN
        ST --> L
        ST --> MI
        ST --> OT
        ST --> TT
        SI --> DA
        SI --> IS
        SI --> IN
        SI --> L
        SI --> MI
        SI --> PR
        SI --> TT
        EC --> IS
        EC --> IN
        EC --> L
        EC --> MI
        EC --> OT
        EC --> ST
        EC --> TT
        SM --> EC
        SM --> IS
        SM --> L
        SM --> MI
        SM --> OT
        SM --> PB
        SM --> SI
        SM --> ST
        SM --> TT
        ME --> EC
        ME --> SI
        SB --> DA
        SB --> L
        SB --> MI
        SB --> SM
        SB --> ST
    end

    subgraph PO["PlanarOptim (optimization layer)"]
        OP[PlanarOptim.jl]
        PL[Plotting]
        OP --> PL
    end

    E --> PC
    PM --> PC
    LM --> PC
    WA --> PC
    RE --> PC
    OP --> O
    OP --> PC

Data Flow Architecture

sequenceDiagram
    participant U as User Strategy
    participant S as Strategy Engine
    participant E as Execution Mode
    participant D as Data Layer
    participant X as Exchange
    
    U->>S: Initialize Strategy
    S->>D: Load Historical Data
    D->>S: Return [OHLCV](guides/../guides/data-management.md#ohlcv-data) Data
    
    loop Trading Loop
        S->>U: Call Strategy Logic
        U->>S: Generate Signals
        S->>E: Execute Orders
        E->>X: Place Orders (Live/Paper)
        X->>E: Order Confirmation
        E->>S: Update Positions
        S->>D: Log Trade Data
    end

Development Setup

Prerequisites

  • Julia 1.11+: Latest stable Julia version
  • Git: With submodule support
  • Python 3.8+: For CCXT integration (managed via CondaPkg)
  • Docker (optional): For containerized development

Initial Setup

  1. Clone Repository:
git clone https://github.com/BubbleParticles/Planar.jl
cd Planar.jl
  1. Environment Setup:
# Enable direnv (recommended)
direnv allow

# Or manually set environment
export JULIA_PROJECT=Planar
export JULIA_NUM_THREADS=$(nproc)
  1. Install Dependencies:

Development Environment

IDE Configuration

VS Code with Julia Extension:

{
    "julia.environmentPath": "Planar",
    "julia.enableTelemetry": false,
    "julia.execution.resultDisplay": "both",
    "julia.lint.run": true,
    "julia.format.indent": 4,
    "julia.format.margin": 92
}

Vim/Neovim with LanguageServer.jl:

require'lspconfig'.julials.setup{
    settings = {
        julia = {
            environmentPath = "Planar",
            format = {
                indent = 4,
                margin = 92
            }
        }
    }
}

PlanarDev Utilities

The PlanarDev package assists developers by providing helper functions for working with Planar and for conducting tests.

Precompilation Control

Package entry files (src/<Package>.jl) follow a uniform pattern: unconditional include("module.jl"), optionally followed by include("precompile.jl") when the package name appears in the JULIA_PRECOMP environment variable:

module Foo
include("module.jl")
if occursin(string(@__MODULE__), get(ENV, "JULIA_PRECOMP", ""))
    include("precompile.jl")
end
end

The JULIA_PRECOMP variable (set in .envrc) determines which packages run precompile workloads. This replaces the old JULIA_NOPRECOMP deferred-loading pattern which has been removed from all source files.

Precompile workloads are driven by PrecompileTools via the JULIA_PRECOMP / JULIA_FULL_PRECOMP environment variables (see .envrc), and sysimages are built with scripts/compile.jl (PackageCompiler) by the Docker planar-sysimage and planar-sysimage-optim targets.

Precompile Workload Restrictions

Precompile workloads must not:

  • Eval into Main (breaks incremental compilation — use $Pkg interpolation instead)
  • Call live HTTP endpoints or start exchange gateways (wrap in try/catch or mock)
  • Contain __revise_mode__ = :eval (belongs in user settings only)
  • Load external strategy modules via @eval parent begin ... using Pkg: Pkg ... end (use $Pkg interpolation)

Method Invalidation Strategy

The order of using ... statements when loading modules can influence method invalidation. To minimize invalidation, arrange the module imports starting with the ones most likely to cause invalidations to the ones least likely. For instance, placing using Python at the beginning can expedite loading times.

Modules known for heavy invalidations:

  • Python
  • Ccxt (initiates the Python async loop)
  • Data (relies on Zarr and DataFrames)
  • Plots (depends on Makie)

To reduce invalidations, include external modules in only one local package and then use that package as a dependency in other local packages. For instance, if DataFrames is a dependency of the local package Data, and you want to use DataFrames in the Stats package, do not add DataFrames to Stats dependencies. Instead, use Data and import DataFrames from there.

Handling Segfaults

In rare cases involving complex multi-threaded scenarios, disable and re-enable the garbage collector (GC) around the loading of Planar to avoid segmentation faults.

Refer to PythonCall.jl issue #201 for more details.

Dependency Management

When adding dependencies, ensure that a dependency is only included in one subpackage. If you need the same dependency in another subpackage, add the first subpackage as the dependency, not the external module.

The order of using or import statements within packages is crucial. Always import external dependencies before internal ones to minimize method invalidations.

See Also

Contribution Guidelines

Code Style and Standards

Julia Formatting

Planar uses JuliaFormatter with Blue style.

Configuration in .JuliaFormatter.toml:

style = "blue"
margin = 92
indent = 4
always_for_in = true
whitespace_typedefs = true
whitespace_ops_in_indices = true
remove_extra_newlines = true

Naming Conventions

  • Types: PascalCase (MyCustomType)
  • Functions: snakecase (`myfunction`)
  • Constants: UPPERCASE (`MYCONSTANT`)
  • Modules: PascalCase (MyModule)
  • Variables: snakecase (`myvariable`)

Documentation Standards

All public functions must have docstrings.

Extension Best Practices

Creating New Modules

  1. Module Structure:
MyNewModule/
├── Project.toml
├── Manifest.toml
├── src/
│   ├── MyNewModule.jl
│   ├── types.jl
│   ├── functions.jl
│   └── utils.jl
├── test/
│   ├── runtests.jl
│   └── test_functions.jl
└── README.md
  1. Project.toml Template:
name = "MyNewModule"
uuid = "generate-new-uuid"
authors = ["Your Name <your.email@example.com>"]
version = "0.1.0"

[deps]
# Only essential dependencies
Engine = {path = "../Engine"}

[compat]
julia = "1.11"
  1. Module Definition:

Performance Optimization Guidelines

  1. Type Stability:

  2. Memory Allocation:

  3. Benchmarking:

Debugging and Profiling

Debugging Strategies

  1. Logging:

  2. Interactive Debugging:

  3. Testing Utilities:

Continuous Integration

GitHub Actions Workflow

# .github/workflows/ci.yml
name: CI
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        julia-version: ['1.11', '1.12']
    
    steps:
    - uses: actions/checkout@v3
      with:
        submodules: recursive
    
    - uses: julia-actions/setup-julia@v1
      with:
        version: ${{ matrix.julia-version }}
    
    - uses: julia-actions/cache@v1
    
    - uses: julia-actions/julia-buildpkg@v1
      with:
        project: Planar
    
    - uses: julia-actions/julia-runtest@v1
      with:
        project: Planar
    
    - uses: julia-actions/julia-processcoverage@v1
    
    - uses: codecov/codecov-action@v3

Release Process

Version Management

  1. Semantic Versioning: Follow SemVer (MAJOR.MINOR.PATCH)

    • MAJOR: Breaking changes
    • MINOR: New features, backward compatible
    • PATCH: Bug fixes, backward compatible

  2. Release Checklist:

    • [ ] Update version in Project.toml
    • [ ] Update CHANGELOG.md
    • [ ] Run full test suite
    • [ ] Update documentation
    • [ ] Create release tag
    • [ ] Build and test Docker images

Community Guidelines

Issue Reporting

When reporting issues, include:

  • Julia version and Planar version
  • Minimal reproducible example
  • Expected vs actual behavior
  • Error messages and stack traces
  • System information (OS, architecture)

Pull Request Process

  1. Fork the repository
  2. Create feature branch from develop
  3. Make changes with tests
  4. Update documentation
  5. Submit pull request with clear description
  6. Address review feedback
  7. Merge after approval

Code Review Standards

  • Functionality: Does the code work as intended?
  • Performance: Are there performance implications?
  • Style: Does it follow project conventions?
  • Tests: Are there adequate tests?
  • Documentation: Is it properly documented?
  • Breaking Changes: Are breaking changes justified and documented?

This comprehensive development guide ensures consistent, high-quality contributions to the Planar ecosystem.