뇌파 헬멧으로 머릿속에서 만든 문장을 39% 오류율까지 읽어낸 AI

무엇을 했나

1. 두피 위에서 자기장을 재는 MEG 장비로 뇌수술 없이 문장을 해독하는 모델 Brain2Qwerty v2를 만들었다.
2. 키를 누르는 순간마다 맞춰 분류하던 기존 방식과 달리, 연속된 뇌 신호에서 실시간으로 글자·단어·문장을 함께 뽑아내는 3단계 구조를 썼다.
3. 글자 단위 인코더, 단어 단위 정렬기, 거대언어모델(LLM) 미세조정을 이어붙여 학습 데이터가 10배 늘어나자 해독 정확도가 데이터 양에 비례해 꾸준히 좋아졌다.
4. AI 코딩 에이전트에게 파이프라인 개선을 맡겼더니 전통적 자동 탐색(Optuna)보다 더 큰 성능 향상을 찾아냈고, 이는 다른 참가자에게도 일반화됐다.
5. 결과적으로 평균 단어 오류율 39%, 최고 참가자 기준 절반 가까운 문장이 한 단어 이하 오류로 정확히 복원됐으며, 이는 이전 비침습 최고기록보다 두 배 가까이 개선된 수치다.

왜 중요한가

말을 하거나 움직일 수 없는 환자들에게 수술 없이 의사소통 수단을 제공할 가능성을 보여준다는 점에서 의미가 크다. 뇌수술이 필요한 침습적 뇌-컴퓨터 인터페이스와의 성능 격차를 데이터 양을 늘리는 것만으로 좁힐 수 있다는 근거를 제시했다.

이 논문의 용어

• MEG(뇌자도) · 두피 밖에서 뇌의 미세한 자기장을 측정해 신경 활동을 읽는 비침습 장비
• CER/WER/SemER · 각각 글자·단어·의미 단위로 해독 결과와 정답이 얼마나 다른지 나타내는 오류율 지표
• CTC(연결주의적 시간 분류) · 글자가 언제 발생했는지 미리 몰라도 연속 신호에서 순서대로 글자를 뽑아내게 학습하는 방법
• LoRA · 거대언어모델 전체를 다시 학습하지 않고 일부 저차원 파라미터만 추가로 학습해 적응시키는 미세조정 기법
• Model Soup · 참가자별로 따로 학습한 모델의 가중치를 평균 내어 하나의 범용 모델로 합치는 방법

본문에 싣지 못한 그림

• Figure 1: Asynchronous MEG decoding is unlocked by recording scale and variety. A. Experimental protocol. Left. We recorded healthy volunteers for 10 hours each using Magnetoencephalography (MEG) while they typed natural sentences they heard a few seconds prior. Right. Average MEG source reconstruction at the time of key press suggest that MEG primarily picks neural activity in the motor cortex. B. Approaches for brain-to-text decoding. Synchronous decoding consists in classifying the character from windows time-locked to each keystroke (e.g. levy2025brain). Asynchronous decoding consists in decoding text from a continuous brain signal, and can thus be applied in real-time, although with some potential delays (e.g. feghhi2025time). C. Quantity (hours per participant) and diversity (number of unique sentences) of our dataset (EnglishBCBL) as compared to levy2025brain (SpanishBCBL). D. Character-error-rate (CER) for the synchronous encoder of levy2025brain. Each coloured dot is one subject; the bar plots the across-subject mean. E. Same as D for our asynchronous encoder. F. Scaling of the asynchronous encoder CER as a function of the amount of training data (log scale), expressed as total recording hours per subject (test set fixed across all points). The blue curve is the across-subject mean on our EnglishBCBL dataset. The orange diamond places SpanishBCBL on the same axis after training our asynchronous encoder on it. G. Impact of sentence-list variety on asynchronous encoder CER at matched total sentence count: 128 unique sentences × 2 repetitions (SpanishBCBL protocol, orange) versus 256 unique sentences (EnglishBCBL protocol, blue). Two-sided Mann-Whitney U test across the n=9 subjects per condition: (***) denotes p < 0.001. Across all panels, CER is computed per sentence, then averaged within each subject, and finally averaged across subjects.
• Figure 2: Brain2Qwerty v2 architecture. Our pipeline is solely input with the continuous MEG recording corresponding to an entire typed sentence and outputs the decoded sentence thanks to three jointly-optimized modules. First, the Encoder is trained with a CTC loss (graves2006connectionist) to extract character-level representations from brain activity, and outputs both MEG Embeddings and a sequence of characters. Second, the Aligner learns, with a SigLIP loss (zhai2023sigmoid), to group and align the MEG embeddings with the true Word Embeddings. Finally, a Large Language Model (LLM) is input with both the MEG tokens and the Encoder’s text to generate the correct sentence autoregressively and with LoRA finetuning (huLoRALowRankAdaptation2021).
• Figure 3: Brain2Qwerty v2 enables word- and meaning-level decoding from MEG. A–C. Per-subject Character (A), Word (B), and Semantic (C) Error Rate for three decoders: Encoder – MEG encoder greedy CTC predictions; Encoder + N-gram – encoder predictions decoded with a 6-gram character language model; Brain2Qwerty v2 – full pipeline conditioned on both CTC text and MEG-derived word embeddings. Each dot is one subject; the black bar is the cross-subject mean, printed above each column. D. Per-sentence word-edit count for the Best, Median, and Worst Subject, sorted ascending; colour-matched dashed lines mark each subject’s perfect-decoded boundary (WER=0, annotated as “% perfect”). E. Decoded sentences from Brain2Qwerty v2 for the Best, Median, and Worst Subjects on three example targets. F. Decoded sentences from the three decoders (Brain2Qwerty v2, Encoder + N-gram, Encoder) on three example targets from the median subject (S01). In A–B, ground-truth sentences are shown next to “Target”; word substitutions and insertions are highlighted in red and deletions are denoted by [-]. Statistical tests in B–D: two-sided paired Wilcoxon signed-rank across n=9 subjects. Brackets show three adjacent decoder comparisons (lower level) plus the outer Encoder vs. Brain2Qwerty v2 comparison (upper level). Significance levels: ∗p<0.05, ∗∗p<0.01, ∗∗∗p<0.001. All annotated comparisons reach p=0.0039 (∗⁣∗), the minimum achievable for n=9 paired observations.
• Figure 4: CTC Tokenizer and tuned LLM adaptation cut WER by 20% over baseline alignments. A. Per-subject word error rate for three contrastive token alignment strategies: Patch Tokenizer (fixed number of patches), Sentence Alignment (one embedding per sentence), CTC Tokenizer (CTC-segmented embeddings based on the predicted space token). B. Word-count calibration of the CTC Tokenizer. C. Example of word-level token retrieval using the CTC Tokenizer strategy. Each column lists the top-5 tokens corresponding to different words of the Qwen3-0.6B vocabulary ranked by cosine similarity to the brain-derived word embedding. A green cell marks the rank at which the ground-truth word is retrieved. D. LoRA rank sweep for Qwen3-0.6B (attention-only targets, α=2​r). Three training strategies are compared across LoRA ranks: All Subjects (single adapter for all subjects), Per-subject (independent adapters per subject), and Model Soup (uniform average of the per-subject model weights). Shaded bands show SEM across the 9 subjects, computed by subtracting each subject’s mean across conditions before taking the standard deviation so that they reflect within-subject variability only. E. LLM backbone scaling and LoRA target-module ablation. Each group shows two bars: All Subjects at r=2 (orange) and Model Soup at r=128 (green). The first three groups use attention-only LoRA with increasing backbone size (Qwen3-0.6B, 1.7B, 4B); the fourth uses Qwen3-4B with LoRA extended to all seven linear projections. Error bars use the same SEM definition than in D. All metrics are sentence-level WER, averaged per subject and then across the 9 subjects. Brackets report paired two-sided Wilcoxon signed-rank tests (p∗<0.05, p∗∗<0.01, p∗⁣∗∗<0.001); non-significant comparisons are unlabelled.
• Figure 5: Autonomous AI coding agents discover configurations that outperform classical optimization. A. Running-best validation WER on Subject S01 over successive experiments. Each dot represents one training run; colored step functions track the cumulative best WER for each method. The dashed grey line indicates the default configuration with only 4 parameters: learning rate, batch size, weight decay and rank with reasonable values. Optuna search is denoted with purple (dash-dotted). It searches within the 4-parameters space. Three independent AutoResearch agents are in blue, orange and green. B. Test WER evaluated on all 9 subjects for each method’s final configuration. Each dot represents one subject’s mean WER; colored horizontal lines indicate the cross-subject mean. Methods are ordered by decreasing (worse) WER from left to right. Significance brackets show paired Wilcoxon signed-rank tests between Optuna and each AutoResearch agent (∗∗∗ p<0.001).
• Figure S1: MEG sources over time relative to keystroke onset.
• Figure S2: tSNE clustering. A. Keyboard color map based on key location. B. tSNE clustering of key representations taken from the last layer of the Conformer of an untrained MEG Encoder, colored by subject (left) and key (right). C. tSNE clustering of key representations taken from the BrainModule (bottom row) and last layer of the Conformer (top row) of our trained MEG Encoder, colored by subject (left) and key (right).
• Figure S3: Decoded sentence examples across difficulty bands. 18 sentences decoded by Brain2Qwerty v2 for the Best, Median, and Worst subjects. Sentences are stratified by their mean Brain2Qwerty v2 WER, averaged across all repetitions across all 9 subjects, into three terciles. Six sentences randomly drawn from each: Best Decoded Sentences (bottom tercile, top row), Median Decoded Sentences (middle tercile, middle row), and Worst Decoded Sentences (top tercile, bottom row). Each prediction is rendered with per-word colour coding: correct words in black, substitutions / insertions in red, and deletions marked [-] in orange. Per-row WER is shown on the right.
• Figure S4: Brain2Qwerty v2 is a neuroLLM, not a corrector of CTC predictions. A–C. Per-subject Character (A), Word (B), and Semantic (C) Error Rate for two configurations of our LLM-based decoder that share the same Qwen3-0.6B backbone. Brain2Qwerty + LLM (grey) conditions the LLM only on the encoder’s predictions; Brain2Qwerty v2 (green) additionally conditions the LLM on the MEG-derived word embeddings (“Neuro Tokens”). Each pair of dots is one subject (n=9); thin black lines connect the same subject across the two configurations. Black horizontal bars are cross-subject means, printed beside each bar. Brackets: two-sided paired Wilcoxon signed-rank, ∗∗p<0.01.
• Figure S5: Encoder CER linearly predicts our model’s performance, and architecture choices set the encoder CER. Y-axis is shared by the three panels and represents the MEG Encoder character error rate (CER) on the test set. A. Per-sentence Brain2Qwerty v2 word error rate (WER) versus MEG Encoder CER, averaged across subjects per unique sentence. The green line is a linear regression. B. Same as (A) with semantic error rate (SemER) on the x-axis. C. Per-subject MEG Encoder CER for three encoder architectures: Temporal Patch Transformer, BrainModule Transformer, and the MEG Encoder used in this paper. All three pairwise comparisons are significant (paired Wilcoxon, two-sided).

최신 논문

• AI 코딩 에이전트에게 과학 소프트웨어 수리를 시켜보니, 절반도 제대로 못 고쳤다AI 코딩 에이전트에게 과학 소프트웨어 수리를 시켜보니, 절반도 제대로 못 고쳤다arXiv:2608.197992026-08-21
• 논문 속 시연이 아니라 실제 서비스에 넣을 수 있는 희소 어텐션 만들기논문 속 시연이 아니라 실제 서비스에 넣을 수 있는 희소 어텐션 만들기arXiv:2608.197582026-08-21
• 고객상담 AI 상담원이 규정을 '한 번의 행동'이 아니라 '전체 절차'로 지키게 만드는 방법고객상담 AI 상담원이 규정을 '한 번의 행동'이 아니라 '전체 절차'로 지키게 만드는 방법arXiv:2608.198612026-08-21
• 로봇 팔에게 사람의 시연 없이 새 일 시키기, 말 잘하는 AI가 대신 가르친다로봇 팔에게 사람의 시연 없이 새 일 시키기, 말 잘하는 AI가 대신 가르친다arXiv:2608.198912026-08-21
• 에이전트 학습용 환경을 새로 만드는 대신, 기존 환경에 '패치 부품'을 씌워 그 에이전트의 약점에 맞게 바꾸는 방법에이전트 학습용 환경을 새로 만드는 대신, 기존 환경에 '패치 부품'을 씌워 그 에이전트의 약점에 맞게 바꾸는 방법arXiv:2608.198802026-08-21
• AI 모델을 '소유'하지 못한 조직은 안전 통제도 절반밖에 못 한다AI 모델을 '소유'하지 못한 조직은 안전 통제도 절반밖에 못 한다arXiv:2608.192162026-08-21
• AI가 선생님 모델을 따라 배우다가, 정답에 다가가는 '좋은 생각'까지 억누르는 문제를 잡아낸다AI가 선생님 모델을 따라 배우다가, 정답에 다가가는 '좋은 생각'까지 억누르는 문제를 잡아낸다arXiv:2608.194082026-08-21
• AI가 특정 사람 말투를 흉내내도록 시켜봤더니, 결국 AI 자신의 말투에서 못 벗어난다AI가 특정 사람 말투를 흉내내도록 시켜봤더니, 결국 AI 자신의 말투에서 못 벗어난다arXiv:2608.197462026-08-21

METAL MEDIA 최신 기사

• TUIDE, 아이슬란드에서 촬영한 K팝 로드무비 'SUN KISS'로 데뷔
• NCT 127, 2년 만의 정규앨범 'BLINGY' 발매