# paper.pdf ## Metadata - PDFFormatVersion=1.6 - Language=en-US - IsLinearized=true - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - CreationDate=D:20260619194019+05'30' - Creator=Adobe InDesign 17.4 (Windows) - ModDate=D:20260619194355+05'30' - Producer=Adobe PDF Library 16.0.7 - Trapped=False - xmpmm:instanceid=uuid:d0ca5846-5dcb-4af5-9f7c-363bfe766da7 - xmpmm:originaldocumentid=xmp.did:A33916180F2068118083ACD1B377E157 - xmpmm:documentid=xmp.id:fe95e0ad-31ae-e944-a027-3dff1ec521a7 - xmpmm:renditionclass=proof:pdf - xmpmm:derivedfrom=xmp.iid:082b993a-97d2-5a42-ba75-23e97596d805xmp.did:082b993a-97d2-5a42-ba75-23e97596d805xmp.did:A33916180F2068118083ACD1B377E157default - xmpmm:history=convertedfrom application/x-indesign to application/pdfAdobe InDesign 17.4 (Windows)/2026-06-19T19:40:19+05:30 - xmp:createdate=2026-06-19T19:40:19+05:30 - xmp:modifydate=2026-06-19T19:43:55+05:30 - xmp:metadatadate=2026-06-19T19:43:55+05:30 - xmp:creatortool=Adobe InDesign 17.4 (Windows) - xmprights:marked=False - photoshop:modifydate=2019-01-31T16:16:25 - photoshop:createdate=2019-01-24T11:43:46 - dc:format=application/pdf - pdf:producer=Adobe PDF Library 16.0.7 - pdf:trapped=False - extensisfontsense:slug=3023672065GraphikOpenType - PS3023672065--0Graphik-Bold1.0011904271029GraphikOpenType - PS1904271029--0Graphik-Regular1.0012428528031News Gothic MT OTOpenType - PS2428528031Monotype Typography0NewsGothicMTOT-Regular1.000543760319News Gothic MT OTOpenType - PS543760319Monotype Typography0NewsGothicMTOT-Bold1.0002785939051KnockoutOpenType - PS2785939051--0Knockout-68FullFeatherwt1.200423856041Graphik X CondOpenType - PS423856041--0GraphikXCond-Super1.0001526035435Graphik X CondOpenType - PS1526035435--0GraphikXCond-Bold1.0003093728306Graphik CondOpenType - PS3093728306--0GraphikCond-Black1.0011341118539Graphik CondOpenType - PS1341118539--0GraphikCond-Regular1.0011385893531GraphikOpenType - PS1385893531--0Graphik-Semibold1.0012986479693Graphik CompactOpenType - PS2986479693--0GraphikCompact-Regular1.0024223138361Graphik CompactOpenType - PS4223138361--0GraphikCompact-Semibold1.0023673047546Harding SevenOpenType - PS3673047546--0HardingSeven-Extrabold0.000103712566Harding SevenOpenType - PS103712566--0HardingSeven-Roman0.0002232452979Harding SevenOpenType - PS2232452979--0HardingSeven-Medium0.000631916388Harding SevenOpenType - PS631916388--0HardingSeven-Bold0.000157629484Publico BannerOpenType - PS157629484--0PublicoBanner-Black1.005 ## Contents ### Page 1 nature neurosciencehttps://doi.org/10.1038/s41593-026-02303-2ArticleNoninvasive decoding of typed sentences from human brain activityIn the format provided by the authors and uneditedSupplementary information ### Page 2 Supplementary Information 10 Motor-topy of MEG SignalsLeft Pinky Left Ring Left MiddleLeft Index Thumbs Right IndexRight Middle Right Ring Right Pinky0.06 0.04 0.02 0.00 0.02 0.04 0.06 0.5 0.0 0.5 Dimension 1 0.50.0 0.5 Dimension 2t r b v f g e c d a z q s w x n m u h y j i k polFigure S1 Finger-specific topographical maps and MDS analysis. (Left) Grand-average sensor topographies showing the spatial distribution of MEG gradiometer activity specific to each finger (n = 19 participants). Each map displays the group-level median difference between the finger-specific evoked response and the overall median response at ≈ 40 ms before keystroke onset. Colour scale: median gradiometer difference (shared across all maps). (Right) Multidimensional Scaling (MDS) projection of the character-level grand-average MEG sensor topographies (dissimilarity = 1 − r, Pearson correlation). Each point represents one character (n = 26letters, excluding space), coloured by the finger used for typing on a standard Spanish QWERTY layout. Convex hulls group characters assigned to the same finger.To verify that our decoding relies on physiological motor signals, we analyzed the intrinsic motor-topy of the raw MEG data. We generated topographical maps for the eight fingers and thumbs. Specifically, to isolate digit-specific activity, we computed the difference between the median response of each finger and the median response across all fingers. Overall, while these MEG topographies tend to peak over the central sensors, the resulting maps remain noisy and difficult to interpret. It is likely due to the fact that participants are not using the typical finger-to-key associations, as it is commonly observed in typists that were not formally trained56. Using the traditional finger-to-key mapping then adds noise to our data. Second, we extracted raw sensor amplitudes at the same timestep and computed the Grand Average evoked response for each character and applied Multidimensional Scaling (MDS) to visualize the similarity of these topographies (Figure S1, right). The results reveal: • Hemispheric Separation: The MDS perfectly separates keystrokes executed with the Left Hand (warm colors) from the Right Hand (cool colors), confirming the signal is grounded in contralateral motor cortex activity. • Finger Somatotopy: Within each hand, we observe a rough clustering of fingers. However, unlike the sharp clustering in our model’s embedding (Figure 6B), the raw signal shows significant overlap between adjacent fingers. Again, these analyses assume standard touch-typing conventions, as precise finger-to-key mapping was not recorded; individual deviations may contribute to the observed overlap. Nevertheless, this confirms that while MEG signals contain fundamental motor-topic information, the raw data is sufficiently noisy that simple linear mapping is difficult. This highlights the necessity of our Brain2Qwerty architecture to refine these coarse inputs into separable character representations. 16 ### Page 3 11 Two-step Decoding and Detection AnalysisEEG MEG 0.0 0.2 0.4 0.6 0.8 F1 Score Brain2Qwerty Two-Step 0.00 0.25 0.50 0.75 1.00 CER 0.648 0.736***Brain2Qwerty Two-Step0.295 0.661***True: l a f u n ci o n l e e l o s a r c h i vo s c on t en id o s Brain2Qwerty: l a f u n ci o na n a l o s a r c h i vo s c on t en id o sTwo-Step: l a f u n c i o n d e l a s c o n e x i o n e sFigure S2 Performance and Example of the Two-Step Decoding Pipeline. (Top Left) F1-Score of the Keystroke Detector for EEG (blue, n = 20 participants) and MEG (green, n = 19participants). Data are presented as mean values ± SEM across participants. Each participant corresponds to an independent biological replicate. Individual participant data points are shown; horizontal lines indicate the mean. EEG: n = 20 participants; MEG: n = 19 participants.Two-sided Wilcoxon signed-rank test. Statistical significance: *p < 0.05, ** p < 0.01, *** p < 0.001. Each participant corresponds to an independent biological replicate. (Bottom) True Sentence, Brain2Qwerty and Two-Step prediction for a fixed sentence. Characters are colored in blue (matches) and red (errors).A limitation of the main framework is its reliance on ground-truth keystroke timestamps. To evaluate the feasibility of asynchronous decoding and quantify the difficulty gap between classification and detection, we implemented a full 2-step pipeline. Specifically: • Detector: We trained a dense neural network similar to our backbone but shallower (depth=4 vs 8) to output a probability of keystroke onset at every time sample. We applied a peak detection algorithm (minimal distance of 50ms between two detected events) to extract estimated timestamps. • Classifier: We utilized the Brain2Qwerty model. Crucially, to handle the inevitable temporal misalign- ment from the detector, we retrained the classifier with temporal jitter augmentation (random shifts of the input window) to improve robustness. Figure S2 show that: • Detection is non-trivial: The detector achieves an F1-score of 0.78 for MEG and 0.76 for EEG. While these metrics appear reasonable, they are insufficient for high-fidelity decoding. • Performance drop: when we replace ground-truth onsets with these detected onsets, the Character Error Rate (CER) deteriorates significantly (two-sided Wilcoxon signed-rank test): MEG: CER increases from 0.295 to 0.661 (p = 3.81 × 10−6). EEG: CER increases from 0.648 to 0.736 (p = 3.81 × 10−6). • Decoded Example: Figure S2 aligns a decoded sentence of both pipelines. The comparison highlights that even small timing shifts in the detection step force the classifier to process misaligned signal windows, leading to cascading character errors (marked in red) that are absent in the synchronized setup. This drastic drop confirms that detecting a keypress non-invasively with sufficient precision is non-trivial. Consequently, bridging the gap from synchronized classification to fully asynchronous decoding represents another important challenge to be solved for this research to be extended to a real-time setting. We hope that the present effort will establish a valuable baseline from which to improve modeling. 17 ### Page 4 12 Analysis of Word-Level LLM Post-ProcessingBrain2Qwerty Brain2Qwerty LLM 0.00 0.25 0.50 0.75 CER0.648 0.664 Brain2Qwerty Brain2Qwerty LLM0.295 0.323Figure S3 Impact of LLM post-processing on decoding performance. Comparison of Character Error Rate (CER) for EEG (left, n = 20 participants) and MEG (right, n = 19 participants) between the Brain2Qwerty model (with N-gram language model) and Brain2Qwerty corrected by GPT-4o-mini. Each dot represents a single participant; horizontal black lines indicate group means. GPT-4o-mini post-processing significantly increased CER for both modalities (two-sided paired Wilcoxon signed-rank test; EEG: p = 2.33 × 10−3; MEG: p = 3.81 × 10−6), indicating that LLM correction did not improve character-level decoding accuracy.To investigate whether incorporating word-level semantic information could improve decoding accuracy, we evaluated a post-processing correction strategy using a general-purpose instruction-tuned Large Language Model (GPT-4o-mini57). We provided the LLM with the outputs from both the Brain2Qwerty model (post N-gram) and the Conv+Trans model (pre N-gram) to maximize context and prompted it to correct character-level errors. This approach resulted in a worsened Character Error Rate (CER). Specifically, CER increased from 0.648 to 0.664 for EEG and from 0.295 to 0.323 for MEG (Figure S3). While the precise reasons for this outcome remain to be clarified, we hypothesize that the N-gram model already corrects the most obvious errors. Furthermore, the isolated nature of the sentences likely prevents the language model from leveraging sufficient context to be effective.13 Sensitivity to Language Model Weight1 2 5 6 8 10 LM Weight 0.65 0.66 0.67 CER1 2 5 6 8 10 LM Weight 0.32 0.34 CERFigure S4 Impact of Language Model Weight (α) on Decoding Performance. Character Error Rate (CER), averaged across participants, is plotted against the Language Model weight for EEG (left, blue, n = 20 participants) and MEG (right, green, n = 19 participants). Each data point represents the mean CER over all participants for a given weight value. The optimal weight (α = 5) is used throughout the main analyses. No error bars are shown as this figure illustrates the parameter selection curve rather than a statistical comparison.To evaluate the influence of the language model on decoding accuracy, we analyzed the Character Error Rate (CER) as a function of the weighting parameter α. This parameter balances the contribution of the brain signals against the linguistic prior (N-gram). As shown in Figure S4, the performance profile exhibits a characteristic convex shape (U-shape) for both MEG and EEG modalities. The error rate is minimized at an optimal weight of α=5. At lower weights (α<5), the system relies too heavily on the raw neural decoder and fails to correct simple typing or phonetic errors (under-correction). Conversely, at higher weights (α>5), the system over-relies 18 ### Page 5 on the language model. Performance degrades here because the decoder starts disregarding strong neural evidence, effectively forcing common N-gram sequences even when the brain signal clearly dictates a different (less probable) character. The optimal value (α=5) represents the spot where the language model acts as a smart editor.14 Language Model Effects Across Decoder Ablations0.0 0.5 1.0 CER 0.842 0.813 ***EEG Linear0.787 0.793EEGNet0.758 0.713 ***ConvLinear Linear N-gram 0.0 0.5 1.0 CER 0.784 0.767 **MEGEEGNet EEGNet N-gram0.726 0.717 Conv Conv N-gram0.505 0.376 ***Figure S5 Language Model Ablations. Character Error Rate (CER) for three baseline decoders (Linear, EEGNet, Conv) with and without N-gram correction, for EEG (top row, n = 20 participants) and MEG (bottom row, n = 19 participants). Each dot represents a single participant; horizontal black lines indicate group means. Significance was assessed with two-sided paired Wilcoxon signed-rank tests: ∗ p < 0.05, ∗∗ p < 0.01, ∗ ∗ ∗ p < 0.001; ns, not significant.To directly evaluate whether the N-gram step helps only once the neural decoder reaches a certain reliability threshold, we conducted an ablation in which our final correction step was added on top of three baseline architectures: Linear, EEGNet, and Conv (n = 20 EEG participants; n = 19 MEG participants). The language model is easily thrown off unless the neural predictions cross a minimum reliability threshold on their own. Figure S5 shows that adding the N-gram model moderately improves performance in some conditions but not others. The ranking between ablations (Linear < EEGNet < Conv < Conv+Trans < Brain2Qwerty) is conserved. For EEG, the Linear model improves significantly (CER 0.842 → 0.813; two-sided paired Wilcoxon signed-rank test, p = 4.77 × 10−5), the Conv model also improves (CER 0.758 → 0.713;p = 2.61 × 10−4), while EEGNet does not change significantly (p = 0.14). These results are consistent with the idea that the LM is more often misled than helped when upstream evidence is too noisy. The most telling result is the convolutional decoder on MEG. When the base Conv model is already relatively strong, adding the N-gram model produces a dramatic improvement: CER drops from 0.505 to 0.376 (two-sided paired Wilcoxon signed-rank test, p = 3.81 × 10−6), which places Conv+N-gram almost exactly at the level of Conv+Transformer and reveals a clear “activation point” where the LM begins to shape the decoding in a meaningful way. This sharp transition strongly suggests the presence of a reliability threshold: below it, the LM amplifies errors; above it, the LM can meaningfully refine predictions.Supplementary References[56] Feit, A. M., Weir, D. & Oulasvirta, A. How we type: Movement strategies and performance in everyday typing. In Proceedings of the 2016 chi conference on human factors in computing systems, 4262–4273 (2016). [57] OpenAI. Gpt-4o system card (2024). URL https://arxiv.org/abs/2410.21276. 2410.21276.19