Publications
2026
-
DataComp-VLM: Improved Open Datasets for Vision-Language Models
DCVLM team
arXiv 2026
pdf |
abstract
Building performant Vision-Language Models (VLMs) requires
carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such
curation strategies. We introduce DataComp for VLMs (DCVLM), a benchmark for controlled data-centric experiments to
improve VLM training. As part of DCVLM, we collect 160 datasets spanning four data types -- image-caption pairs,
multimodal interleaved documents, text-only, and instruction-tuning data -- into a corpus of 6T multimodal tokens.
DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B-8B models
and 6.25B-200B token budgets. Models are then evaluated on a carefully selected suite of up to 52 downstream benchmarks
across 9 domains. We conduct extensive experiments on DCVLM and find that data mixing, not filtering, is key to a
high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening
at larger scales. The resulting dataset, DCVLM-Baseline, enables training an 8B VLM to 63.6% accuracy on our 33-task
core suite with 200B training tokens. Compared to FineVision, the state-of-the-art open VLM training dataset, this
represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available at
https://www.datacomp.ai/dcvlm/.
-
VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
Jean Mercat, Sedrick Keh, Kushal Arora, Isabella Huang, Paarth Shah, Haruki Nishimura, Shun Iwase, Katherine Liu
arXiv 2026
pdf |
abstract
We present VLA Foundry, an open-source framework that
unifies LLM, VLM, and VLA training in a single codebase. Most open-source VLA efforts specialize on the action training
stage, often stitching together incompatible pretraining pipelines. VLA Foundry instead provides a shared training
stack with end-to-end control, from language pretraining to action-expert fine-tuning. VLA Foundry supports both
from-scratch training and pretrained backbones from Hugging Face. To demonstrate the utility of our framework, we
train and release two types of models: the first trained fully from scratch through our LLM-->VLM-->VLA pipeline and
the second built on the pretrained Qwen3-VL backbone. We evaluate closed-loop policy performance of both models on
LBM Eval, an open-data, open-source simulator. We also contribute usability improvements to the simulator and the
STEP analysis tools for easier public use. In the nominal evaluation setting, our fully-open from-scratch model is
on par with our prior closed-source work and substituting in the Qwen3-VL backbone leads to a strong multi-task
table top manipulation policy outperforming our baseline by a wide margin.
-
SkillFactory: Self-Distillation for Learning Cognitive Behaviors
Zayne Sprague, Jack Lu, Manya Wadhwa, Sedrick Keh, Mengye Ren, Greg Durrett
ICLR 2026
pdf |
abstract
Reasoning models leveraging long chains of thought
employ various cognitive skills, such as verification of their answers, backtracking, retrying by an alternate method,
and more. Previous work has shown that when a base language model exhibits these skills, training that model further
with reinforcement learning (RL) can learn to leverage them. How can we get models to leverage skills that aren't
exhibited by base models? Our work, SkillFactory, is a method for fine-tuning models to roughly learn these skills
during a supervised fine-tuning (SFT) stage prior to RL. Our approach does not rely on distillation from a stronger
model, but instead uses samples from the model itself, rearranged to provide training data in the format of those
skills. These "silver" SFT traces may be imperfect, but are nevertheless effective for priming a model to acquire
skills during RL. Our evaluation shows that (1) starting from SkillFactory SFT initialization helps a model to
generalize to harder variants of a task post-RL, despite lower performance pre-RL; (2) cognitive skills are indeed
used by the model; (3) RLed SkillFactory models are more robust to regression on out-of-domain tasks than RLed base
models. Our work suggests that inductive biases learned prior to RL help models learn robust cognitive skill use.
-
OpenThoughts: Data Recipes for Reasoning Models
Etash Guha*, Ryan Marten*, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, et al. (OpenThoughts team)
ICLR 2026
pdf |
abstract
Reasoning models have made rapid progress on many
benchmarks involving math, code, and science. Yet, there are still many open questions about the best training
recipes for reasoning since state-of-the-art models often rely on proprietary datasets with little to no public
information available. To address this, the goal of the OpenThoughts project is to create open-source datasets for
training reasoning models. After initial explorations, our OpenThoughts2-1M dataset led to OpenThinker2-32B, the
first model trained on public reasoning data to match DeepSeek-R1-Distill-32B on standard reasoning benchmarks such
as AIME and LiveCodeBench. We then improve our dataset further by systematically investigating each step of our data
generation pipeline with 1,000+ controlled experiments, which led to OpenThoughts3. Scaling the pipeline to 1.2M
examples and using QwQ-32B as teacher yields our OpenThoughts3-7B model, which achieves state-of-the-art results:
53% on AIME 2025, 51% on LiveCodeBench 06/24-01/25, and 54% on GPQA Diamond -- improvements of 15.3, 17.2, and 20.5
percentage points compared to the DeepSeek-R1-Distill-Qwen-7B. All of our datasets and models are available on
https://openthoughts.ai.
-
A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
TRI LBM team
Science Robotics 2026
pdf |
abstract |
website
Robot manipulation has seen tremendous progress in recent
years, with imitation learning policies enabling successful performance of dexterous and hard-to-model tasks.
Concurrently, scaling data and model size has led to the development of capable language and vision foundation
models, motivating large-scale efforts to create general-purpose robot foundation models. While these models have
garnered significant enthusiasm and investment, meaningful evaluation of real-world performance remains a challenge,
limiting both the pace of development and inhibiting a nuanced understanding of current capabilities. In this paper,
we rigorously evaluate multitask robot manipulation policies, referred to as Large Behavior Models (LBMs), by
extending the Diffusion Policy paradigm across a corpus of simulated and real-world robot data. We propose and
validate an evaluation pipeline to rigorously analyze the capabilities of these models with statistical confidence.
We compare against single-task baselines through blind, randomized trials in a controlled setting, using both
simulation and real-world experiments. We find that multi-task pretraining makes the policies more successful and
robust, and enables teaching complex new tasks more quickly, using a fraction of the data when compared to
single-task baselines. Moreover, performance predictably increases as pretraining scale and diversity grows.
2025
-
Should VLMs be Pre-trained with Image Data?
Sedrick Keh*, Jean Mercat*, Samir Yitzhak Gadre, Kushal Arora, Igor Vasiljevic, Benjamin Burchfiel, Shuran Song, Russ Tedrake, Thomas Kollar, Ludwig Schmidt, Achal Dave
ICLR 2025
pdf |
abstract
Pre-trained LLMs that are further trained with image data perform
well on vision-language tasks. While adding images during a second training phase effectively unlocks this capability,
it is unclear how much of a gain or loss this two-step pipeline gives over VLMs which integrate images earlier into the
training process. To investigate this, we train models spanning various datasets, scales, image-text ratios, and amount
of pre-training done before introducing vision tokens. We then fine-tune these models and evaluate their downstream
performance on a suite of vision-language and text-only tasks. We find that pre-training with a mixture of image and
text data allows models to perform better on vision-language tasks while maintaining strong performance on text-only
evaluations. On an average of 6 diverse tasks, we find that for a 1B model, introducing visual tokens 80% of the way
through pre-training results in a 2% average improvement over introducing visual tokens to a fully pre-trained model.
-
Improving Test-Time Search for LLMs with Backtracking Against In-Context Value Verifiers
Anikait Singh, Kushal Arora, Sedrick Keh, Jean Mercat, Tatsunori Hashimoto, Chelsea Finn, Aviral Kumar
ICLR 2025 Workshop on Reasoning and Planning for LLMs
openreview
2024
-
DataComp-LM: In search of the next generation of training sets for language models
DCLM team
NeurIPS Datasets and Benchmarks 2024
pdf |
abstract
We introduce DataComp for Language Models (DCLM), a testbed for
controlled dataset experiments with the goal of improving language models. As part of DCLM, we provide a standardized
corpus of 240T tokens extracted from Common Crawl, effective pretraining recipes based on the OpenLM framework, and a
broad suite of 53 downstream evaluations. Participants in the DCLM benchmark can experiment with data curation strategies
such as deduplication, filtering, and data mixing at model scales ranging from 412M to 7B parameters. As a baseline for
DCLM, we conduct extensive experiments and find that model-based filtering is key to assembling a high-quality training
set. The resulting dataset, DCLM-Baseline enables training a 7B parameter language model from scratch to 64% 5-shot
accuracy on MMLU with 2.6T training tokens. Compared to MAP-Neo, the previous state-of-the-art in open-data language
models, DCLM-Baseline represents a 6.6 percentage point improvement on MMLU while being trained with 40% less compute.
Our baseline model is also comparable to Mistral-7B-v0.3 and Llama 3 8B on MMLU (63% & 66%), and performs similarly on
an average of 53 natural language understanding tasks while being trained with 6.6x less compute than Llama 3 8B. Our
results highlight the importance of dataset design for training language models and offer a starting point for further
research on data curation.
-
SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages
SEACrowd team
EMNLP 2024
pdf |
abstract
Southeast Asia (SEA) is a region rich in linguistic
diversity and cultural variety, with over 1,300 indigenous languages and a population of 671 million people.
However, prevailing AI models suffer from a significant lack of representation of texts, images, and audio datasets
from SEA, compromising the quality of AI models for SEA languages. Evaluating models for SEA languages is challenging
due to the scarcity of high-quality datasets, compounded by the dominance of English training data, raising concerns
about potential cultural misrepresentation. To address these challenges, we introduce SEACrowd, a collaborative
initiative that consolidates a comprehensive resource hub that fills the resource gap by providing standardized
corpora in nearly 1,000 SEA languages across three modalities. Through our SEACrowd benchmarks, we assess the quality
of AI models on 36 indigenous languages across 13 tasks, offering valuable insights into the current AI landscape in
SEA. Furthermore, we propose strategies to facilitate greater AI advancements, maximizing potential utility and
resource equity for the future of AI in SEA.
-
Linearizing Large Language Models
Jean Mercat*, Igor Vasiljevic*, Sedrick Keh*, Kushal Arora, Achal Dave, Adrien Gaidon, Thomas Kollar
COLM 2024
pdf |
abstract
Linear transformers have emerged as a subquadratic-time
alternative to softmax attention and have garnered significant interest due to their fixed-size recurrent state
that lowers inference cost. However, their original formulation suffers from poor scaling and underperforms
compute-matched transformers. Recent linear models such as RWKV and Mamba have attempted to address these
shortcomings by proposing novel time-mixing and gating architectures, but pre-training large language models
requires significant data and compute investments. Thus, the search for subquadratic architectures is limited by
the availability of compute and quality pre-training datasets. As a cost-effective alternative to pre-training
linear transformers, we propose Scalable UPtraining for Recurrent Attention (SUPRA). We present a method to
uptrain existing large pre-trained transformers into Recurrent Neural Networks (RNNs) with a modest compute budget.
This allows us to leverage the strong pre-training data and performance of existing transformer LLMs, while requiring
5% of the training cost. We find that our linearization technique leads to competitive performance on standard
benchmarks, but we identify persistent in-context learning and long-context modeling shortfalls for even the largest
linear models. Our code and models can be found at https://github.com/TRI-ML/linear_open_lm.
-
Language Models Scale Reliably with Over-training and on Downstream Tasks
Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, Suchin Gururangan, Mitchell Wortsman, Rulin Shao, Jean Mercat, Alex Fang, Jeffrey Li, Sedrick Keh, Rui Xin, Marianna Nezhurina, Igor Vasiljevic, Jenia Jitsev, Alexandros G Dimakis, Gabriel Ilharco, Shuran Song, Thomas Kollar, Yair Carmon, Achal Dave, Reinhard Heckel, Niklas Muennighoff, Ludwig Schmidt
ICLR 2025
pdf |
abstract
Scaling laws are useful guides for developing
language models, but there are still gaps between current scaling studies and how language models are ultimately
trained and evaluated. For instance, scaling is usually studied in the compute-optimal training regime
(i.e., "Chinchilla optimal" regime); however, in practice, models are often over-trained to reduce inference costs.
Moreover, scaling laws mostly predict loss on next-token prediction, but ultimately models are compared based on
downstream task performance. In this paper, we address both shortcomings. To do so, we create a testbed of 104 models
with 0.011B to 6.9B parameters trained with various numbers of tokens on three data distributions. First, we investigate
scaling in the over-trained regime. We fit scaling laws that extrapolate in both the number of model parameters and the
ratio of training tokens to parameters. This enables us to predict the validation loss of a 1.4B parameter, 900B token
run (i.e., 32x over-trained) and a 6.9B parameter, 138B token run -- each from experiments that take 300x
less compute. Second, we relate the perplexity of a language model to its downstream task performance via a power law.
We use this law to predict top-1 error averaged over downstream tasks for the two aforementioned models using experiments
that take 20x less compute. Our experiments are available at https://github.com/mlfoundations/scaling.
-
A Critical Evaluation of AI Feedback for Aligning Large Language Models
Archit Sharma, Sedrick Keh, Eric Mitchell, Chelsea Finn, Kushal Arora, Thomas Kollar
NeurIPS 2024
pdf |
abstract
Reinforcement learning with AI feedback (RLAIF) is a popular
paradigm for improving the instruction-following abilities of powerful pre-trained language models. RLAIF first performs
supervised fine-tuning (SFT) using demonstrations from a teacher model and then further fine-tunes the model with
reinforcement learning (RL), using feedback from a critic model. While recent popular open-source models have
demonstrated substantial improvements in performance from the RL step, in this paper we question whether the complexity
of this RL step is truly warranted for AI feedback. We show that the improvements of the RL step are virtually entirely
due to the widespread practice of using a weaker teacher model (e.g. GPT-3.5) for SFT data collection than the critic
(e.g., GPT-4) used for AI feedback generation. Specifically, we show that simple supervised fine-tuning with GPT-4 as
the teacher outperforms existing RLAIF pipelines. More generally, we find that the gains from RLAIF vary substantially
across base model families, test-time evaluation protocols, and critic models. Finally, we provide a mechanistic
explanation for when SFT may outperform the full two-step RLAIF pipeline as well as suggestions for making RLAIF
maximally useful in practice.
-
Where It Really Matters: Few-Shot Environmental Conservation Media Monitoring for Low-Resource Languages
Sameer Jain, Sedrick Scott Keh, Shova Chhetri, Karun Dewan, Pablo Izquierdo, Johanna Prussmann, Pooja Shrestha, César Suárez, Zheyuan Ryan Shi, Lei Li, Fei Fang
AAAI 2024
pdf |
abstract
Environmental conservation organizations
routinely monitor news content on conservation in protected areas to maintain situational awareness of developments
that can have an environmental impact. Existing automated media monitoring systems require large amounts of data
labeled by domain experts, which is only feasible at scale for high-resource languages like English. However, such
tools are most needed in the global south where news of interest is mainly in local low-resource languages, and far
fewer experts are available to annotate datasets sustainably. In this paper, we propose NewsSerow, a method to
automatically recognize environmental conservation content in low-resource languages. NewsSerow is a pipeline of
summarization, in-context few-shot classification, and self-reflection using large language models (LLMs). Using at
most 10 demonstration example news articles in Nepali, NewsSerow significantly outperforms other few-shot methods
and achieves comparable performance with models fully fine-tuned using thousands of examples. The World Wide Fund
for Nature (WWF) has deployed NewsSerow for media monitoring in Nepal, significantly reducing their operational
burden, and ensuring that AI tools for conservation actually reach the communities that need them the most. NewsSerow
has also been deployed for countries with other languages like Colombia.
2023
-
Asking More Informative Questions for Grounded Retrieval
Sedrick Keh, Justin T Chiu, Daniel Fried
NAACL Findings 2024
pdf |
abstract
When a model is trying to gather information in an interactive setting,
it benefits from asking informative questions. However, in the case of a grounded multi-turn image identification task, previous
studies have been constrained to polar yes/no questions, limiting how much information the model can gain in a single turn.
We present an approach that formulates more informative, open-ended questions. In doing so, we discover that off-the-shelf
visual question answering (VQA) models often make presupposition errors, which standard information gain question selection
methods fail to account for. To address this issue, we propose a method that can incorporate presupposition handling into both
question selection and belief updates. Specifically, we use a two-stage process, where the model first filters out images which
are irrelevant to a given question, then updates its beliefs about which image the user intends. Through self-play and human
evaluations, we show that our method is successful in asking informative open-ended questions, increasing accuracy over the past
state-of-the-art by 14%, while resulting in 48% more efficient games in human evaluations.
-
Doolittle: Benchmarks and Corpora for Academic Writing Formalization
Shizhe Diao, Yongyu Lei, Liangming Pan, Tianqing Fang, Wangchunshu Zhou, Sedrick Keh, Min-Yen Kan, Tong Zhang
EMNLP 2023
pdf |
abstract
Improving the quality of academic writing is a meaningful but
challenging task. Conventional methods of language refinement focus on narrow, specific linguistic features within isolated
sentences, such as grammatical errors and improper word use. We propose a more general task, Academic Writing Formalization (AWF),
to improve the overall quality of formal academic writing at the paragraph level. We formulate this language refinement task as a
formal text style transfer task which transfers informal-academic text to formal-academic and contribute a large-scale non-parallel
dataset, Doolittle, for this purpose. Concurrently, we apply a method named metric-oriented reinforcement learning (MORL) to two
large language models (LLM) where we incorporate different levels of automatic feedback into the training process. Our experiments
reveal that existing text transfer models and grammatical error correction models address certain aspects of AWF but still have a
significant performance gap compared to human performance. Meanwhile, language models fine-tuned with our MORL method exhibit
considerably improved performance, rivaling the latest chatbot ChatGPT, but still have a non-negligible gap compared to the ground
truth formal-academic texts in Doolittle.
-
Hashtag-Guided Low-Resource Tweet Classification
Shizhe Diao*, Sedrick Scott Keh*, Liangming Pan, Zhiliang Tian, Yan Song, Tong Zhang
WWW 2023
pdf |
abstract
Social media classification tasks (e.g., tweet sentiment analysis,
tweet stance detection) are challenging because social media posts are typically short, informal, and ambiguous. Thus, training
on tweets is challenging and demands large-scale human-annotated labels, which are time-consuming and costly to obtain. In this paper,
we find that providing hashtags to social media tweets can help alleviate this issue because hashtags can enrich short and ambiguous
tweets in terms of various information, such as topic, sentiment, and stance. This motivates us to propose a novel Hashtag-guided
Tweet Classification model (HashTation), which automatically generates meaningful hashtags for the input tweet to provide useful
auxiliary signals for tweet classification. To generate high-quality and insightful hashtags, our hashtag generation model retrieves
and encodes the post-level and entity-level information across the whole corpus. Experiments show that HashTation achieves significant
improvements on seven low-resource tweet classification tasks, in which only a limited amount of training data is provided, showing that
automatically enriching tweets with model-generated hashtags could significantly reduce the demand for large-scale human-labeled data.
Further analysis demonstrates that HashTation is able to generate high-quality hashtags that are consistent with the tweets and their
labels. The code is available at https://github.com/shizhediao/HashTation.
2022
-
PANCETTA: Phoneme Aware Neural Completion to Elicit Tongue Twisters Automatically
Sedrick Scott Keh, Steven Y. Feng*, Varun Gangal*, Malihe Alikhani, Eduard Hovy
EACL 2023
pdf |
abstract |
github
Tongue twisters are meaningful sentences that are difficult to
pronounce. The process of automatically generating tongue twisters is challenging since the generated utterance must satisfy
two conditions at once: phonetic difficulty and semantic meaning. Furthermore, phonetic difficulty is itself hard to characterize
and is expressed in natural tongue twisters through a heterogeneous mix of phenomena such as alliteration and homophony.
In this paper, we propose PANCETTA: Phoneme Aware Neural Completion to Elicit Tongue Twisters Automatically. We leverage
phoneme representations to capture the notion of phonetic difficulty, and we train language models to generate original tongue
twisters on two proposed task settings. To do this, we curate a dataset called PANCETTA, consisting of existing English tongue
twisters. Through automatic and human evaluation, as well as qualitative analysis, we show that PANCETTA generates novel,
phonetically difficult, fluent, and semantically meaningful tongue twisters.
-
EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation
Sedrick Scott Keh*, Rohit Bharadwaj*, Emmy Liu**, Simone Tedeschi**, Varun Gangal, Roberto Navigli
EMNLP 2022 FigLang Workshop
pdf |
abstract |
github |
slides
We introduce EUREKA, an ensemble-based approach to perform automatic
euphemism detection. We (1) identify and correct potentially mislabelled rows in the dataset, (2) curate an expanded corpus
called EuphAug, (3) leverage model representations of Potentially Euphemistic Terms (PETs), and (4) explore using representations
of semantically close sentences to aid in classification. Using these methods, EUREKA was able to achieve state-of-the-art results
on the public leaderboard of the Euphemism Detection Shared Task, with a macro-F1 score of 0.881. Our code is available at
this https URL.
-
Exploring Euphemism Detection in Few-Shot and Zero-Shot Settings
Sedrick Scott Keh
EMNLP 2022 FigLang Workshop
pdf |
abstract |
github |
poster
This work builds upon the Euphemism Detection Shared Task proposed
in the EMNLP 2022 FigLang Workshop, and extends it to few-shot and zero-shot settings. We demonstrate a few-shot and zero-shot
formulation using the dataset from the shared task, and we conduct experiments in these settings using RoBERTa and GPT-3. Our
results show that language models are able to classify euphemistic terms relatively well even on new terms unseen during training,
indicating that it is able to capture higher-level concepts related to euphemisms.
-
PINEAPPLE: Personifying INanimate Entities by Acquiring Parallel Personification Data for Learning Enhanced Generation
Sedrick Scott Keh, Kevin Lu, Steven Y. Feng*, Varun Gangal*, Harsh Jhamtani, Malihe Alikhani, Eduard Hovy
COLING 2022
pdf |
abstract |
github |
talk |
slides |
poster
A personification is a figure of speech that endows inanimate entities
with properties and actions typically seen as requiring animacy. In this paper, we explore the task of personification generation.
To this end, we propose PINEAPPLE: Personifying INanimate Entities by Acquiring Parallel Personification data for Learning Enhanced
generation. We curate a corpus of personifications called PersonifCorp, together with automatically generated de-personified
literalizations of these personifications. We demonstrate the usefulness of this parallel corpus by training a seq2seq model to
personify a given literal input. Both automatic and human evaluations show that fine-tuning with PersonifCorp leads to significant
gains in personification-related qualities such as animacy and interestingness. A detailed qualitative analysis also highlights key
strengths and imperfections of PINEAPPLE over baselines, demonstrating a strong ability to generate diverse and creative
personifications that enhance the overall appeal of a sentence.
-
NewsPanda: Media Monitoring for Timely Conservation Action
Sedrick Scott Keh*, Zheyuan Ryan Shi*, David J. Patterson, Nirmal Bhagabati, Karun Dewan, Areendran Gopala, Pablo Izquierdo, Debojyoti Mallick, Ambika Sharma, Pooja Shrestha, Fei Fang
IAAI 2023
pdf |
abstract |
github |
website
Non-governmental organizations for environmental conservation have a
significant interest in monitoring conservation-related media and getting timely updates about infrastructure construction projects
as they may cause massive impact to key conservation areas. Such monitoring, however, is difficult and time-consuming. We introduce
NewsPanda, a toolkit which automatically detects and analyzes online articles related to environmental conservation and
infrastructure construction. We fine-tune a BERT-based model using active learning methods and noise correction algorithms to
identify articles that are relevant to conservation and infrastructure construction. For the identified articles, we perform further
analysis, extracting keywords and finding potentially related sources. NewsPanda has been successfully deployed by the World Wide
Fund for Nature teams in the UK, India, and Nepal since February 2022. It currently monitors over 80,000 websites and 1,074 conservation
sites across India and Nepal, saving more than 30 hours of human efforts weekly. We have now scaled it up to cover 60,000 conservation
sites globally.
2021 and older
-
Semi-supervised Noisy Student Pre-training on EfficientNet Architectures for Plant Pathology Classification
Sedrick Scott Keh
arXiv 2020
pdf |
abstract
In recent years, deep learning has vastly improved the identification and
diagnosis of various diseases in plants. In this report, we investigate the problem of pathology classification using images of a single
leaf. We explore the use of standard benchmark models such as VGG16, ResNet101, and DenseNet 161 to achieve a 0.945 score on the task.
Furthermore, we explore the use of the newer EfficientNet model, improving the accuracy to 0.962. Finally, we introduce the state-of-the-art
idea of semi-supervised Noisy Student training to the EfficientNet, resulting in significant improvements in both accuracy and convergence
rate. The final ensembled Noisy Student model performs very well on the task, achieving a test score of 0.982.
-
Myers-Briggs Personality Classification and Personality-specific Language Generation Using Pre-trained Language Models
Sedrick Scott Keh, I-Tsun Cheng
arXiv 2019
pdf |
abstract
The Myers-Briggs Type Indicator (MBTI) is a popular personality metric that uses
four dichotomies as indicators of personality traits. This paper examines the use of pre-trained language models to predict MBTI personality
types based on scraped labeled texts. The proposed model reaches an accuracy of 0.47 for correctly predicting all 4 types and 0.86 for correctly
predicting at least 2 types. Furthermore, we investigate the possible uses of a fine-tuned BERT model for personality-specific language generation.
This is a task essential for both modern psychology and for intelligent empathetic systems.
|
|
