Skip to content

TeraChem Protocol Buffers (TCPB)

TCPB is TeraChem's general-purpose client-server interface for embedding TeraChem inside an external driver. Unlike the [Single Point Engine Mode (MPI)] (single-point-engine.md), which is a custom MPI-named-port protocol, TCPB exchanges Google Protocol Buffer messages over an ordinary TCP socket. This makes the same TeraChem server reachable from a Python, C++, Fortran, or any other client on the same node or on a remote machine, without requiring the client to link MPI.

The interface and the freely-available tcpb-python and tcpb-cpp client libraries are described in Cruzeiro, Wang, Pieri, Hohenstein, Martinez, J. Chem. Phys. 158, 044801 (2023)1, which also reports the performance benefit over a file-based QM/MM driver: TeraChem keeps its GPU state and SCF guess in memory between requests, so a typical QM/MM MD step is \(>2\times\) faster than restarting the binary on every step.

Starting a TeraChem TCPB server

A TeraChem build that includes protocol-buffer support (the default in recent builds) is launched as a server with the -s / --server flag, which takes the TCP port number as its argument:

terachem -s 12345          # bind to TCP port 12345
terachem -g 01 -s 12345    # use GPUs 0 and 1

The server stays alive after each job, holding the GPU context and (when requested) the previous wavefunction so the next request starts from a warm guess. Stop it by sending the process a signal (Ctrl-C in the terminal that launched it). One server handles one client at a time; for multiple simultaneous drivers, launch one server per port.

Protocol

The wire protocol is a fixed 8-byte header (two big-endian int32s) followed by a serialized protobuf payload:

Bytes Field Meaning
0-3 MessageType 0 = STATUS, 1 = MOL, 2 = JOBINPUT, 3 = JOBOUTPUT
4-7 size Length of the serialized protobuf body, in bytes
8… payload Protobuf-encoded message of the declared type

A single job consists of three message types in this order:

  1. Client → server: JobInput (geometry, method, basis, run type, MM atoms if any, user options).
  2. Server → client: Status messages — first an accepted (or busy if the server is occupied), then working while the job runs, then completed when the result is ready. The client polls by sending an empty header of type STATUS.
  3. Server → client: JobOutput (energy, gradient, charges, dipoles, MO files on disk, and method-specific extras such as CIS dipoles, NACMEs, MM gradients).

The full schema, including all available fields, lives in commbox/src/terachem_server.proto in the TeraChem source tree. Highlights:

  • JobInput.run — ENERGY (0), GRADIENT (1), COUPLING (14), TDCI (16), CI_VEC_OVERLAP (19).
  • JobInput.method — most TeraChem methods are available as enums (HF, B3LYP, PBE0, \(\omega\)PBEh, …). For anything not in the enum, set method_str to the keyword string.
  • JobInput.user_options — a flat list of alternating "key", "value" strings. Any TeraChem keyword that isn't a first-class protobuf field can be passed through this list (e.g. ["maxit", "500", "guess", "hcore", "purify", "no"]).
  • JobInput.qmmm_type — NO_QMMM, POINT_CHARGE, or TC_OPENMM. POINT_CHARGE sends mmatom_position and mmatom_charge arrays; TC_OPENMM sends an Amber prmtop file (path or inlined content) plus zero-based qm_indices.
  • JobInput.md_global_type — NORMAL (default; rebuild everything), NEW_CONDITION (rebuild but keep global state alive after the job), or CONTINUE (reuse global state and the previous orbital guess; only the QM/MM coordinates and MM charges may change). For MD, the standard pattern is one NEW_CONDITION job followed by many CONTINUE jobs.

Protocol Buffers recommends keeping each message under 1 MB, which is satisfied by all standard QM/MM workloads.

Client libraries

Two reference clients are maintained by the Martínez group:

  • tcpb-python — pure Python. Provides a high-level TCProtobufClient context manager whose compute_job_sync(jobtype, geom, units, **options) method handles the input protobuf, polling, and unpacking of the output for you. Almost every example in this manual uses it.
  • tcpb-cpp — C++ client. Builds a shared library that exposes a clean C++ API; a thin Python wrapper (pytcpb) is built alongside. This client is used by Amber's TeraChem QM/MM driver and is the recommended path for Fortran/C++ host codes.

Both packages ship the generated protobuf bindings, so a host program does not need to learn the protobuf framework directly.

Examples

The following examples are drawn from tests/tests/TCPB/ in the TeraChem source tree and exercise the same code paths used by the integration test suite. Each script reads the TeraChem server's port from sys.argv[1]; in practice you would set it to whatever you used when starting terachem -s.

Energy + gradient on a single geometry

A CASCI(2,2)/6-31G** gradient on the ground state of ethene. All options that are not first-class protobuf fields (fon, closed, active, …) ride along through the high-level wrapper.

ethene_gradient_0/tcpb_test.py
from tcpb import TCProtobufClient

atoms = ['C', 'C', 'H', 'H', 'H', 'H']
geom = [ 0.35673483, -0.05087227, -0.47786734,
         1.61445821, -0.06684947, -0.02916681,
        -0.14997206,  0.87780529, -0.62680155,
        -0.16786485, -0.95561368, -0.69426370,
         2.15270896,  0.84221076,  0.19314809,
         2.16553127, -0.97886933,  0.15232587]

with TCProtobufClient(host='localhost', port=12345) as TC:
    options = {
        'method':       'hf',
        'basis':        '6-31g**',
        'atoms':        atoms,
        'charge':       0,
        'spinmult':     1,
        'closed_shell': True,
        'restricted':   True,

        'precision':    'double',
        'threall':      1e-20,

        'casci':        'yes',
        'fon':          'yes',
        'closed':       7,
        'active':       2,
        'cassinglets':  3,

        'castarget':     0,
        'castargetmult': 1,
    }
    results = TC.compute_job_sync('gradient', geom, 'angstrom', **options)
    print(results['energy'])
    print(results['gradient'])

Non-adiabatic coupling between two CASCI states

The coupling job type returns the energies of the two requested states, the transition dipole, and the non-adiabatic coupling matrix element (NACME). tvarnac selects the time-derivative coupling formulation.

ethene_coupling_01/tcpb_test.py
with TCProtobufClient(host='localhost', port=12345) as TC:
    options = {
        'method': 'hf', 'basis': '6-31g**',
        'atoms': atoms, 'charge': 0, 'spinmult': 1,
        'closed_shell': True, 'restricted': True,
        'precision': 'double', 'threall': 1e-20,

        'casci': 'yes', 'fon': 'yes',
        'closed': 7, 'active': 2, 'cassinglets': 3,

        'nacstate1': 0,
        'nacstate2': 1,
        'castargetmult': 1,
        'tvarnac': 'yes',
    }
    results = TC.compute_job_sync('coupling', geom, 'angstrom', **options)
    energy = results['energy']
    tdip   = results['cas_transition_dipole']
    nacme  = results['nacme']

CI-vector overlap between two CASCI calculations

The ci_vec_overlap job type computes wave-function overlaps between two CASCI calculations at potentially different geometries; it is the primitive used in TeraChem-driven nonadiabatic dynamics. Run the two CASCI jobs first (reading the CI vectors and orbital coefficients off disk via the job_scr_dir reported in the output), then issue a ci_vec_overlap job referencing both sets of files.

ethylene_overlap_11/tcpb_test.py
from os import path
from tcpb import TCProtobufClient

with TCProtobufClient(host='localhost', port=12345) as TC:
    base = dict(method='hf', basis='6-31g**', atoms=atoms,
                charge=0, spinmult=1, closed_shell=True, restricted=True,
                precision='double', threall=1e-20,
                casci='yes', closed=5, active=6, cassinglets=5)

    casci_opts = dict(directci='yes', dcimaxiter=40,
                      dcisigmamethod='kh', caswritevecs='yes')

    # geometry 1
    r1 = TC.compute_job_sync('energy', geom,  'angstrom', **dict(base, **casci_opts))
    cvec1 = path.join(r1['job_scr_dir'], 'CIvecs.Singlet.dat')
    orb1  = path.join(r1['job_scr_dir'], 'c0')

    # geometry 2
    r2 = TC.compute_job_sync('energy', geom2, 'angstrom', **dict(base, **casci_opts))
    cvec2 = path.join(r2['job_scr_dir'], 'CIvecs.Singlet.dat')
    orb2  = path.join(r2['job_scr_dir'], 'c0')

    overlap_opts = dict(geom2=geom2,
                        cvec1file=cvec1, cvec2file=cvec2,
                        orb1afile=orb1, orb2afile=orb2)
    r = TC.compute_job_sync('ci_vec_overlap', geom, 'angstrom',
                            **dict(base, **overlap_opts))
    print(r['ci_overlap'])

Point-charge QM/MM (B3LYP/6-31G)

A QM water in a field of three MM point charges (a second water). The MM coordinates and charges are sent in atomic units. After the initial NEW_CONDITION call, subsequent steps with CONTINUE reuse the global state and the previous SCF guess — this is the speedup that makes TCPB-driven QM/MM MD practical.

qmmm_2water/tcpb_test.py (lower-level protobuf API)
import terachem_server_qmmm_pb2 as pb

job = pb.JobInput()
job.mol.xyz.extend(geom_qm)               # in bohr
job.mol.units = pb.Mol.UnitType.Value('BOHR')
job.mol.atoms.extend(['O', 'H', 'H'])
job.mol.charge = 0
job.mol.multiplicity = 1
job.mol.restricted = True
job.mol.closed = True

job.md_global_type = pb.JobInput.MDGlobalTreatment.Value('NEW_CONDITION')
job.run            = pb.JobInput.RunType.Value('GRADIENT')
job.method         = pb.JobInput.MethodType.Value('B3LYP')
job.basis          = '6-31g'

job.user_options.extend(['maxit', '500', 'guess', 'hcore',
                         'scf', 'diis', 'precision', 'double',
                         'threall', '1e-12', 'convthre', '1e-12'])

job.qmmm_type = pb.JobInput.QmmmType.Value('POINT_CHARGE')
job.mmatom_position.extend(geom_mm)        # bohr
job.mmatom_charge.extend([-0.51966, 0.25983, 0.25983])

The JobOutput returned by the server populates gradient (QM atoms), mmatom_gradient (MM atoms), charges, dipoles, and the path to the saved orbital file in orb1afile.

OpenMM/Amber-driven QM/MM

For Amber-style QM/MM the prmtop file is sent inline (or by path) along with the zero-indexed list of QM atoms; TeraChem builds the MM force field internally through its OpenMM driver:

qmmm_2water_openmm/tcpb_test.py (excerpt)
with open('2water.prmtop') as f:
    prmtop_content = f.read()

job.qmmm_type      = pb.JobInput.QmmmType.Value('TC_OPENMM')
job.prmtop_content = prmtop_content
job.qm_indices.extend([0, 1, 2])    # the QM water

This is the exact path the Amber QM/MM driver takes in its TeraChem integration1, which is shipped as &tc namelist support in AmberTools (see QM/MM → OpenMM/Amber driver).

Implicit solvent (PCM)

Anything controlled by a TeraChem keyword can be passed through user_options (low-level API) or as a plain Python keyword argument (high-level API). For example, a COSMO single-point gradient on benzene:

benzene_pcm/tcpb_test.py
with TCProtobufClient(host='localhost', port=12345) as TC:
    options = dict(method='b3lyp', basis='6-31g*',
                   atoms=atoms, charge=0, spinmult=1,
                   closed_shell=True, restricted=True,
                   precision='double', threall=1e-12, purify='no',
                   pcm='cosmo', epsilon=2.274)
    results = TC.compute_job_sync('gradient', geom, 'bohr', **options)

Interactive MD orbital tracking

Interactive MD (IMD) uses the same TCPB transport with a couple of extra fields in the JobInput to carry the previous-step molecular orbitals and geometry forward, so the new SCF starts from the previous orbitals rotated into the new basis. The relevant fields are imd_initial_orbital, imd_orbital_type, imd_xyz_previous, and imd_mo_previous; the test under IMD_ethene_basic/ is the canonical example.

Choosing TCPB vs. the MPI single-point engine

Use TCPB if the host code is in any language other than C/C++/Fortran-with-MPI, if the client runs on a different machine from TeraChem, or if the host already speaks protobuf. Use the Single Point Engine Mode (MPI) when the host is itself an MPI application (e.g. legacy nanoreactor/FMS-style drivers) that wants to embed TeraChem inside its own MPI communicator. Either interface yields the same energies and gradients; the choice is one of transport.

Test directory

The integration tests under tests/tests/TCPB/ in the TeraChem source tree are the most up-to-date set of worked TCPB drivers. They cover:

  • HF/CASCI gradients (ethene_gradient_0/1/2)
  • CASCI non-adiabatic couplings (ethene_coupling_01/02/12)
  • CASCI vector overlaps (ethylene_overlap_11/12)
  • PCM (benzene_pcm)
  • Point-charge QM/MM (qmmm_2water, qmmm_water_variable_mm_size)
  • OpenMM-driven QM/MM (qmmm_2water_openmm)
  • Interactive MD (IMD_*) for CASCI, FOMO-SCF, density, Hessian, and MECI driving
  • Nanoreactor with FON annealing (nanoreactor_CH3O-O_fon_anneal)

  1. V. W. D. Cruzeiro, Y. Wang, E. Pieri, E. G. Hohenstein, T. J. Martínez, J. Chem. Phys. 158, 044801 (2023). doi:10.1063/5.0130886. ↩↩