Skip to content
QuantLabTickDirect

DocsResources

Python sample

A single-file sample that authenticates and receives ticks, for checking your connection.

One file (freqrelay_client.py) covers authentication with your API key, subscription, Heartbeat, and reading ticks.

Download freqrelay_client.py

Requirements

  • Python 3.10 or later
  • PyJWT (uses cryptography for Ed25519 signatures)
  • The key ID (ggg_...) and private key (PEM file) of an API key issued on the ggguilds contract page
pip install "pyjwt[crypto]"

Usage

python freqrelay_client.py --server tokyo-alpha-1.tick.quantlab.fi:17000 \
  --key-id ggg_xxxxxxxx --key-file ggguilds-api-key.pem \
  --symbols EURUSD,USDJPY
Authenticated
Subscribed to 2 symbols: EURUSD, USDJPY
EURUSD bid=1.08529 ask=1.08540 upstream=09:41:08.108Z
USDJPY bid=151.01200 ask=151.02500 upstream=09:41:08.112Z
...
datagrams=992 ticks=1041 lost=0 delivery-latency p50=65us p99=140us
OptionMeaning
--serverhost:port of the Tick server
--key-idKey ID of the API key
--key-filePrivate key of the API key (PKCS#8 PEM)
--symbolsSymbols to subscribe to (comma-separated, up to 64). The default is EURUSD,USDJPY
--durationNumber of seconds to receive. 0 (the default) means until Ctrl+C
--quietPrint only the summary every 5 seconds, not each tick
  • Every 5 seconds, the sample prints the number of datagrams and ticks received, the losses counted from gaps in the sequence numbers (lost), and the delivery time (delivery-latency). The delivery time is the time from when the Tick server received the tick until this program received it, and includes the clock difference between the two machines.
  • Prices are printed rounded to 5 decimal places.
  • When you stop it with Ctrl+C, the sample sends an empty Subscribe to end the subscription. The IP address slot is freed immediately.

Troubleshooting

When authentication is rejected, the sample prints the reason (see Result codes for the meanings).

  • UnknownKey: If you just issued the API key, wait about a minute. The sample retries every 10 seconds.
  • InvalidSignature: The key ID and the private key do not belong together.
  • TokenExpired: This machine's clock is off. Sync it with NTP.
  • IpLimitReached: The plan's IP address limit has been reached. Once you stop the connection on another IP address, you can connect immediately (or after 5 seconds, depending on how it was stopped).
  • Nothing is printed: A firewall is blocking UDP, or the host name or port is wrong.

Using it in your own program

  • Keep using one UDP socket from authentication to receiving ticks.
  • Send Heartbeat every second, and when HeartbeatAck tells you the subscription has ended, start over from Hello.
  • Python is slow per message, so to process every symbol with low latency, use a separate receive thread or implement the client in another language.

Source

Show freqrelay_client.py
#!/usr/bin/env python3
"""QuantLab TickDirect の Tick サーバーに接続して、ティックを表示するサンプル。

接続を確かめるための最小の実装です。仕様は https://quantlab.fi/docs を参照してください。

    pip install "pyjwt[crypto]"
    python freqrelay_client.py --server tokyo-alpha-1.tick.quantlab.fi:17000 \\
        --key-id ggg_... --key-file ggguilds-api-key.pem --symbols EURUSD,USDJPY

Ctrl+C で止めると、空の Subscribe を送って購読を終わらせます(IP 上限の枠がすぐに空きます)。
"""

from __future__ import annotations

import argparse
import base64
import socket
import struct
import sys
import time
from dataclasses import dataclass, field

import jwt  # PyJWT

# ---- SBE(freqrelay-sbe.xml、スキーマ ID 1、version 1)。すべてリトルエンディアン ----

SCHEMA_ID = 1
SCHEMA_VERSION = 1

SUBSCRIBE = 1
SUBSCRIBE_ACK = 2
HEARTBEAT = 3
HEARTBEAT_ACK = 4
TICKS = 5
HELLO = 6
CHALLENGE = 7
AUTHENTICATE = 8
AUTHENTICATE_ACK = 9

HEADER = struct.Struct("<HHHH")  # blockLength, templateId, schemaId, version
GROUP = struct.Struct("<HH")  # blockLength, numInGroup
TICK = struct.Struct("<Hqqqq")  # symbolId, bid, ask, upstreamTime, receiveTime

PRICE_SCALE = 100_000_000  # 価格は × 10^8 の固定小数点
NULL_SYMBOL_ID = 0xFFFF
MAX_DATAGRAM = 1200

AUTHENTICATE_RESULTS = [
    "Accepted", "InvalidCookie", "MalformedToken", "InvalidSignature", "TokenExpired", "AudienceMismatch",
    "UnknownKey", "KeyRevoked", "KeyExpired", "PlanNotAccepted", "IpLimitReached", "EndpointLimitReached",
    "ServerFull", "RateLimited", "NotReady",
]
END_REASONS = ["None", "Unknown", "HeartbeatTimeout", "Unsubscribed", "KeyRevoked", "KeyExpired", "Replaced"]
UPSTREAM_STATUSES = ["Disconnected", "Connecting", "Connected"]

# 再試行しても直らない拒否。設定を見直す必要がある
FATAL_RESULTS = {"MalformedToken", "InvalidSignature", "AudienceMismatch", "KeyRevoked", "KeyExpired", "PlanNotAccepted"}


def header(template_id: int, block_length: int) -> bytes:
    return HEADER.pack(block_length, template_id, SCHEMA_ID, SCHEMA_VERSION)


def encode_hello() -> bytes:
    # 32 バイトの 0 の詰め物。サーバーは、Challenge より小さい Hello には応えない
    return header(HELLO, 32) + bytes(32)


def encode_authenticate(request_id: int, token: str) -> bytes:
    data = token.encode("ascii")
    return header(AUTHENTICATE, 4) + struct.pack("<IH", request_id, len(data)) + data


def encode_subscribe(subscriber_token: bytes, request_id: int, symbols: list[str]) -> bytes:
    names = b"".join(symbol.encode("utf-8").ljust(16, b"\0")[:16] for symbol in symbols)
    return header(SUBSCRIBE, 20) + subscriber_token + struct.pack("<I", request_id) + GROUP.pack(16, len(symbols)) + names


def encode_heartbeat(subscriber_token: bytes) -> bytes:
    return header(HEARTBEAT, 16) + subscriber_token


def read_symbol_name(field_bytes: bytes) -> str:
    return field_bytes.split(b"\0", 1)[0].decode("utf-8", errors="replace")


def make_jwt(key_id: str, private_key: bytes, audience: str, cookie: bytes) -> str:
    """ggguilds の標準の JWT に、Challenge の cookie を nonce として足したもの。"""
    now = int(time.time())
    return jwt.encode(
        {
            "iss": key_id,
            "aud": audience,
            "iat": now,
            "exp": now + 300,
            "nonce": base64.urlsafe_b64encode(cookie).rstrip(b"=").decode("ascii"),
        },
        private_key,
        algorithm="EdDSA",
    )


# ---- クライアント ----


@dataclass
class Stats:
    datagrams: int = 0
    ticks: int = 0
    lost: int = 0
    expected_sequence: int = 1
    latencies_us: list[float] = field(default_factory=list)


class Client:
    RETRY_SECONDS = 1.0
    HEARTBEAT_SECONDS = 1.0

    def __init__(self, server: tuple[str, int], key_id: str, private_key: bytes, audience: str, symbols: list[str], quiet: bool):
        self.server = server
        self.key_id = key_id
        self.private_key = private_key
        self.audience = audience
        self.symbols = symbols
        self.quiet = quiet
        # 認証から受信まで、1 つのソケット(送信元のポート)を使い続ける
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, 4 << 20)
        self.sock.connect(server)
        self.request_id = 0
        self.subscriber_token: bytes | None = None
        self.symbol_names: dict[int, str] = {}
        self.stats = Stats()

    def send(self, message: bytes) -> None:
        self.sock.send(message)

    def receive(self, timeout: float) -> tuple[int, memoryview, memoryview] | None:
        """1 つ受け取る。(templateId, ルートブロック, ルートブロックの後ろ) を返す。"""
        # settimeout(0) はソケットを待たないモードにして、データが無いと BlockingIOError になる。0 以下は短い待ちにそろえる
        self.sock.settimeout(max(timeout, 0.001))
        try:
            data = memoryview(self.sock.recv(65536))
        except (socket.timeout, BlockingIOError):
            return None
        except ConnectionRefusedError:
            # Tick サーバーのポートが閉じている(再起動中など)。ICMP の到達不能で recv がすぐ失敗するので、
            # 応答がなかったときと同じだけ待ってから返す(すぐ送り直して要求を連発しないため)
            time.sleep(max(timeout, 0.0))
            return None
        if len(data) < HEADER.size:
            return None
        block_length, template_id, schema_id, _ = HEADER.unpack_from(data)
        if schema_id != SCHEMA_ID or len(data) < HEADER.size + block_length:
            return None
        root = data[HEADER.size:HEADER.size + block_length]
        return template_id, root, data[HEADER.size + block_length:]

    def wait_for(self, template_id: int, timeout: float) -> tuple[memoryview, memoryview] | None:
        """指定した種類のメッセージを待つ。その間に届いた Ticks は処理する。"""
        deadline = time.monotonic() + timeout
        while (remaining := deadline - time.monotonic()) > 0:
            message = self.receive(remaining)
            if message is None:
                return None
            received_id, root, rest = message
            if received_id == template_id:
                return root, rest
            if received_id == TICKS:
                self.on_ticks(root, rest)
        return None

    def connect(self) -> None:
        """Hello → Challenge → Authenticate → Subscribe。応答がなければ送り直す。"""
        while True:
            self.subscriber_token = None
            self.send(encode_hello())
            challenge = self.wait_for(CHALLENGE, self.RETRY_SECONDS)
            if challenge is None:
                continue
            cookie = bytes(challenge[0][:16])

            self.request_id += 1
            self.send(encode_authenticate(self.request_id, make_jwt(self.key_id, self.private_key, self.audience, cookie)))
            ack = self.wait_for(AUTHENTICATE_ACK, self.RETRY_SECONDS)
            if ack is None:
                continue
            _, result = struct.unpack_from("<IB", ack[0])
            result_name = AUTHENTICATE_RESULTS[result] if result < len(AUTHENTICATE_RESULTS) else str(result)
            if result_name != "Accepted":
                if result_name in FATAL_RESULTS:
                    raise SystemExit(f"Authentication failed: {result_name} (see https://quantlab.fi/docs/codes)")
                print(f"Authentication rejected: {result_name}; retrying", file=sys.stderr)
                time.sleep(2 if result_name != "UnknownKey" else 10)
                continue
            self.subscriber_token = bytes(ack[0][5:21])
            print("Authenticated", file=sys.stderr)

            if self.subscribe():
                return

    def subscribe(self) -> bool:
        for _ in range(5):
            self.request_id += 1
            self.send(encode_subscribe(self.subscriber_token, self.request_id, self.symbols))
            ack = self.wait_for(SUBSCRIBE_ACK, self.RETRY_SECONDS)
            if ack is None:
                continue
            root, rest = ack
            _, result = struct.unpack_from("<IB", root)
            if result != 0:
                print("Subscribe rejected: UnknownSubscriber; authenticating again", file=sys.stderr)
                return False
            entry_length, count = GROUP.unpack_from(rest)
            for i in range(count):
                offset = GROUP.size + i * entry_length
                (symbol_id,) = struct.unpack_from("<H", rest, offset)
                name = read_symbol_name(bytes(rest[offset + 2:offset + 18]))
                if symbol_id == NULL_SYMBOL_ID:
                    print(f"Unknown symbol: {name}", file=sys.stderr)
                else:
                    self.symbol_names[symbol_id] = name
            print(f"Subscribed to {len(self.symbol_names)} symbols: {', '.join(self.symbol_names.values())}", file=sys.stderr)
            return True
        return False

    def on_ticks(self, root: memoryview, rest: memoryview) -> None:
        (sequence,) = struct.unpack_from("<Q", root)
        stats = self.stats
        if sequence > stats.expected_sequence:
            stats.lost += sequence - stats.expected_sequence
        if sequence >= stats.expected_sequence or sequence == 1:
            stats.expected_sequence = sequence + 1
        stats.datagrams += 1

        now_ns = time.time_ns()
        entry_length, count = GROUP.unpack_from(rest)
        for i in range(count):
            symbol_id, bid, ask, upstream_ns, receive_ns = TICK.unpack_from(rest, GROUP.size + i * entry_length)
            stats.ticks += 1
            # 到達遅延(サーバーの受信時刻から、ここで受け取るまで。2 台の時計のずれを含む)
            stats.latencies_us.append((now_ns - receive_ns) / 1000)
            if not self.quiet:
                upstream = time.strftime("%H:%M:%S", time.gmtime(upstream_ns / 1e9)) + f".{upstream_ns % 1_000_000_000 // 1_000_000:03d}"
                print(f"{self.symbol_names.get(symbol_id, symbol_id)} bid={bid / PRICE_SCALE:.5f} ask={ask / PRICE_SCALE:.5f} upstream={upstream}Z")

    def run(self, duration: float) -> None:
        self.connect()
        end = time.monotonic() + duration if duration > 0 else float("inf")
        next_heartbeat = time.monotonic()
        next_report = time.monotonic() + 5
        while time.monotonic() < end:
            now = time.monotonic()
            if now >= next_heartbeat:
                self.send(encode_heartbeat(self.subscriber_token))
                next_heartbeat = now + self.HEARTBEAT_SECONDS
            if now >= next_report:
                self.report()
                next_report = now + 5

            message = self.receive(max(0.0, min(next_heartbeat, next_report) - time.monotonic()))
            if message is None:
                continue
            template_id, root, rest = message
            if template_id == TICKS:
                self.on_ticks(root, rest)
            elif template_id == HEARTBEAT_ACK:
                subscribed, upstream, reason = struct.unpack_from("<BBB", root)
                if not subscribed:
                    print(f"Subscription ended: {END_REASONS[reason] if reason < len(END_REASONS) else reason}; connecting again", file=sys.stderr)
                    self.connect()
                elif upstream != 2:
                    print(f"Upstream LP: {UPSTREAM_STATUSES[upstream] if upstream < len(UPSTREAM_STATUSES) else upstream}", file=sys.stderr)

    def report(self) -> None:
        stats = self.stats
        latencies = sorted(stats.latencies_us)
        p50 = latencies[len(latencies) // 2] if latencies else 0.0
        p99 = latencies[min(len(latencies) - 1, len(latencies) * 99 // 100)] if latencies else 0.0
        print(f"datagrams={stats.datagrams} ticks={stats.ticks} lost={stats.lost} delivery-latency p50={p50:.0f}us p99={p99:.0f}us", file=sys.stderr)
        stats.latencies_us.clear()

    def close(self) -> None:
        if self.subscriber_token is not None:
            # 空の Subscribe で購読を終わらせ、IP 上限の枠をすぐに空ける
            self.request_id += 1
            self.send(encode_subscribe(self.subscriber_token, self.request_id, []))
        self.sock.close()


def main() -> None:
    parser = argparse.ArgumentParser(description="FreqRelay のティックを受け取って表示する")
    parser.add_argument("--server", required=True, help="Tick サーバーのアドレスとポート(host:port)")
    parser.add_argument("--key-id", required=True, help="APIキーのキー ID(ggg_...)")
    parser.add_argument("--key-file", required=True, help="APIキーの秘密鍵(PKCS#8 の PEM)")
    parser.add_argument("--symbols", default="EURUSD,USDJPY", help="購読するシンボル(カンマ区切り、64 個まで)")
    parser.add_argument("--audience", default="freqrelay", help="JWT の aud")
    parser.add_argument("--duration", type=float, default=0, help="受け取る秒数(0 なら Ctrl+C まで)")
    parser.add_argument("--quiet", action="store_true", help="ティックを 1 件ずつ表示しない(5 秒ごとの集計だけ)")
    args = parser.parse_args()

    host, _, port = args.server.rpartition(":")
    with open(args.key_file, "rb") as f:
        private_key = f.read()
    symbols = [symbol.strip() for symbol in args.symbols.split(",") if symbol.strip()]

    client = Client((socket.gethostbyname(host), int(port)), args.key_id, private_key, args.audience, symbols, args.quiet)
    try:
        client.run(args.duration)
    except KeyboardInterrupt:
        pass
    finally:
        client.report()
        client.close()


if __name__ == "__main__":
    main()