hyperparameter prompt MLP
hyperparameter based HuggingFace implementation for layer reinforcement.
- Input
- 4976-dim embedding
- Encoder
- 46 x MLP with 42 heads
- Output
- f1 projection
Training config
optimizer=AdamW, lr=0.906, scheduler=exponential, warmup=1128