Floating-Point Determinism

Precision management across hashfita and crustybike. Both projects prioritize deterministic reproducibility over raw performance — a deliberate design choice given the research nature of the work.

---

Precision Inventory

Layer	Precision	Project	Rationale
MLX tensors	mx.float32	Both	Explicit dtype=mx.float32 on every tensor creation
OpenCL output	float (32-bit)	Both	All output buffers use np.float32
Metal kernels	float	crustybike	Metal float maps to IEEE 754 single
OpenCL normalization	(float)(reg >> 16) / 65535.0f	hashfita	Maps 32-bit register to [0,1]
NumPy bridge	np.float32	Both	Forced by PyOpenCL requirements

No float16, bfloat16, or mixed precision anywhere in either codebase.

---

The float32 Opportunity

Apple M3 Pro has native bfloat16 support in the Metal GPU. MLX supports mx.bfloat16 as a dtype. Using bfloat16 for training would:

• Double throughput — bfloat16 operations are 2× faster on M3 Pro Metal
• Halve memory — 2 bytes per element vs 4 bytes
• Minimal accuracy loss — bfloat16 has the same exponent range as float32 (8 bits), only losing mantissa precision (7 bits vs 23 bits)

For the RingTransformer (ring-transformer) with ~272k parameters, this would cut model memory from ~1.1MB to ~550KB and potentially double training throughput.

Risk: The cyclic loss function 1 - cos(theta_pred - theta_actual) is sensitive to precision near theta=0. Cosine near 1.0 requires high mantissa precision to distinguish small differences. bfloat16's 7-bit mantissa may cause gradient vanishing near the optimum.

Recommendation: Mixed precision — bfloat16 for forward pass and activations, float32 for loss computation and gradient accumulation.

---

Deterministic Quantization (hashfita)

hashfita implements a deterministic float→int64 conversion for cross-platform reproducibility:

# quantization.py
QUANTIZATION_SCALE = 1_000_000

def quantize(value: float) -> int:
    return int(np.trunc(value * QUANTIZATION_SCALE))

This is not neural network quantization — it's a deterministic serialization strategy. By converting floats to scaled integers, hashfita ensures that the same computation produces identical results on Apple Silicon and x86_64, avoiding IEEE 754 rounding differences across architectures.

---

PHI as Rational (hashfita)

# constants.py
PHI_NUM = 1618033988749895   # 15 digits of φ
PHI_DEN = 1_000_000_000_000_000  # 10^15

The golden ratio φ is stored as an integer numerator/denominator pair, not as a float. This eliminates floating-point ambiguity in the PHIN Scatter algorithm (ring-transformer) where ((i * PHI) % 1.0) must produce exactly reproducible sequences.

Using integer arithmetic: offset = ((i * PHI_NUM) % PHI_DEN) / PHI_DEN — deterministic on any platform.

---

OpenCL Kernel Precision

Register Normalization (hashfita)

// extract_all_rounds kernel
float normalized = (float)(reg_val >> 16) / 65535.0f;  // [0, 1]

Right-shifts by 16 bits to discard low-order noise, then divides by 65535 (2^16 - 1) to normalize. This is a lossy compression — the bottom 16 bits of each SHA-256 register are discarded. This is acceptable because the high-order bits carry the most entropy in SHA-256's avalanche effect.

DAA Distance (crustybike)

The cartesian_miner kernel computes hash-to-target ratio as a float. This is inherently an approximation — the ratio of two 256-bit integers cannot be exactly represented in float32. The approximation is sufficient for ranking (which hash is closer to target) but not for exact comparison.

---

Cross-Platform Determinism

Technique	File	Purpose
PHI as rational	hashfita/constants.py	Golden ratio reproducibility
int64 quantization	hashfita/quantization.py	Cross-arch float serialization
explicit dtype everywhere	Both projects	No implicit type promotion
integer division for normalization	OpenCL kernels	Deterministic bit manipulation

---

Open Questions

1. Should bfloat16 be tested on the RingTransformer training loop?
2. Can the cyclic loss be reformulated to be bfloat16-safe?
3. Would mx.bfloat16 for inference + mx.float32 for training be a viable mixed-precision strategy?
4. Is the int64 quantization still needed if both platforms use IEEE 754?

---

Cross-References

• compute-optimization — Hub
• ring-transformer — Where precision matters most (cyclic loss)
• muonclip-optimizer — Newton-Schulz orthogonality sensitivity to precision
• mlx-opencl-bridge — Precision at the transfer boundary