skillbase/hummingbot-bot-lifecycle

## Deployment Checklist

Execute in order. If any step fails, stop — do not deploy with unverified conditions.

### 1. Verify infrastructure

```

configure_server()  → confirm connected

```

### 2. Verify exchange connectivity

```

setup_connector(action="setup")  → list connected exchanges

get_portfolio_overview(include_balances=True)  → confirm balances visible

```

### 3. Verify the config exists and looks correct

```

manage_controllers(action="describe", config_name="my_config")

```

Cross-check: spread > 2x fee? Order amount < 5% of book depth? Inventory skew enabled?

### 4. Check for conflicts

```

manage_bots(action="status")  → any bot already trading this pair?

get_portfolio_overview(include_active_orders=True)  → any orphan orders on this pair?

```

### 5. Deploy

```

manage_bots(

  action="deploy",

  bot_name="descriptive-name",

  controllers_config=["my_config"],

  account_name="master_account",

  max_global_drawdown_quote=500,          # always set this

  max_controller_drawdown_quote=200       # always set this

)

```

### 6. Post-deploy verification (within 60 seconds)

```

manage_bots(action="status", bot_name="descriptive-name")  → status = running

get_portfolio_overview(include_active_orders=True)  → orders visible for the pair

```

If orders don't appear within 60 seconds:

```

manage_bots(action="logs", bot_name="descriptive-name", log_type="error", limit=20)

```

## Monitoring Runbook

### Routine checks

| What | How | Healthy signal |

|------|-----|---------------|

| Bot running | `manage_bots(action="status")` | All bots show "running" |

| Orders active | `get_portfolio_overview(include_active_orders=True)` | Orders exist for each active pair |

| Fills happening | `manage_executors(action="search", status="completed", limit=10)` | Recent fills within last hour |

| Inventory balanced | `get_portfolio_overview(include_balances=True)` | Not >70% skewed to one side |

| No errors | `manage_bots(action="logs", bot_name="...", log_type="error", limit=5)` | Empty or only stale errors |

| P&L positive | `get_portfolio_overview()` + `search_history(data_type="orders")` | Spread P&L > inventory drift |

### Warning signs and responses

**No fills for >1 hour** (on a pair with >$50K daily volume)

→ Orders are mispriced or exchange has issues

→ Check: `get_market_data(data_type="order_book", ...)` — is our spread competitive?

→ Check: `manage_bots(action="logs", ..., log_type="error")` — order rejections?

**Inventory skewing >70% one side**

→ Market is trending against us

→ Check: `get_market_data(data_type="candles", ..., interval="1h", days=1)` — trend visible?

→ Action: if strong trend, stop controller: `manage_bots(action="stop_controllers", bot_name="...", controller_names=["the_config"])`

**Bot in error state**

→ Check: `manage_bots(action="logs", bot_name="...", log_type="error", limit=50)`

→ Common causes: exchange connectivity loss, insufficient balance, rate limit exceeded

→ If transient (connectivity): bot usually auto-recovers. Monitor.

→ If persistent: `manage_bots(action="stop_bot", bot_name="...")`, fix cause, redeploy.

**Executor count growing but no fills**

→ Executors being created but never matching — likely config issue

→ Check: `manage_executors(action="search", status="active", limit=20)` — what state are they in?

→ Action: stop bot, review config parameters (spread too wide? order amount too small?)

## Incident Response

### Emergency stop procedure

**When to trigger:**

- Drawdown limit hit (manage_bots auto-stops, but verify)

- Exchange announces unexpected maintenance

- Flash crash or market anomaly

- Bot placing orders at obviously wrong prices

**Procedure:**

```

# 1. Stop bot

manage_bots(action="stop_bot", bot_name="the-bot")

# 2. Verify no active orders remain

get_portfolio_overview(include_active_orders=True)

# 3. If orphan orders exist, stop executors

manage_executors(action="stop", executor_id="...")

# 4. Snapshot state for analysis

get_portfolio_overview()

search_history(data_type="orders", limit=50)

```

**Do NOT restart until:**

- Root cause is identified from logs

- Market conditions are reviewed

- Parameters are adjusted if needed

### Post-incident analysis

```

# What happened — timeline from logs

manage_bots(action="logs", bot_name="...", log_type="all", limit=100)

# What was the financial impact

search_history(data_type="orders", start_time=incident_start, end_time=incident_end)

get_portfolio_overview()

# Was it a strategy issue or operational issue?

# Strategy: bad params (spread too tight, no kill condition) → redesign

# Operational: exchange downtime, connectivity → no action needed, resume when clear

```

## Operational Rules

- **Always set drawdown limits** on deploy — `max_global_drawdown_quote` is the last line of defense

- **One pair per bot** — avoid running multiple bots on the same pair/exchange

- **Verify after every stop** — check `get_portfolio_overview(include_active_orders=True)` for orphan orders

- **Selective control** — use `stop_controllers`/`start_controllers` to toggle individual strategies within a bot without killing the whole bot

- **Runtime config changes** — use `manage_bots(action="update_config")` for running bots, not `manage_controllers(action="upsert")` which is design-time only