Can Reinforcement Learning Adapt Where Optimal Control Cannot? Quantum State Control Under Unknown and Growing Noise

Description

Quantum control is difficult because real qubits are never isolated. They lose energy and phase to their environment, and the exact noise is rarely known in advance. Standard optimal-control methods such as GRAPE and CRAB design a pulse offline using a model of that noise. They work well when the model is right, but the pulse is then fixed. Reinforcement learning (RL) offers a different route: an agent learns a control policy by interacting with a simulated quantum system, with no explicit model of the dynamics. A previous single-qubit study trained a Proximal Policy Optimization (PPO) agent to flip a qubit from |0⟩ to |1⟩ under Lindblad decoherence, comparing it against a constant and a Gaussian pulse. That work shows the framework is feasible, but in a setting where RL has little to prove. The noise is fixed, known and simple, so there is nothing to adapt to and a well-designed optimal-control baseline should match any learned policy. The open question is where RL actually earns its place. In this project the student will first validate and tighten the single-qubit baselines, checking that the reference pulses reach near-perfect fidelity without noise, comparing methods at equal control energy, and adding a GRAPE benchmark. Once the foundation is solid, the student can ask the central question: does RL give an advantage over optimal control when the noise is unknown, drifting, or growing with system size? This matters for several reasons. Real hardware noise is time-varying, device-specific and correlated across qubits, and models of it go stale. A controller that adapts could reduce recalibration. The Bloch-sphere description that works for one qubit also does not scale, since n qubits need 4ⁿ−1 numbers and measurement collapses the state, so multi-qubit RL requires rethinking what the agent observes, perhaps through measured expectation values or a learned encoder. Finally, device-specific noise (T1, T2) connects naturally to the idea of a hardware fingerprint from quantum PUF research, which opens the question of whether pulses learned on one device transfer to another. The project can proceed in stages: (1) single-qubit validation and fair baselines; (2) controlled noise uncertainty, with T1, T2 and detuning varied across episodes, and evaluation on noise different from that seen in training; (3) extension to two qubits with coupling, targeting a Bell state and then a CNOT gate; (4) correlated noise and crosstalk. Evaluation would go beyond final fidelity to include robustness, recovery after a noise change, and pulse energy. Whether noise growth with qubit number favors RL is a hypothesis to test, since robust optimal control may perform equally well, and an honest comparison is itself a result. This is a project for an advanced undergraduate or graduate student in electrical or computer engineering, physics or computer science, with an interest in quantum computing and machine learning.