Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs
arxiv(2024)
摘要
Reasoning encompasses two typical types: deductive reasoning and inductive
reasoning. Despite extensive research into the reasoning capabilities of Large
Language Models (LLMs), most studies have failed to rigorously differentiate
between inductive and deductive reasoning, leading to a blending of the two.
This raises an essential question: In LLM reasoning, which poses a greater
challenge - deductive or inductive reasoning? While the deductive reasoning
capabilities of LLMs, (i.e. their capacity to follow instructions in reasoning
tasks), have received considerable attention, their abilities in true inductive
reasoning remain largely unexplored. To delve into the true inductive reasoning
capabilities of LLMs, we propose a novel framework, SolverLearner. This
framework enables LLMs to learn the underlying function (i.e., y = f_w(x)),
that maps input data points (x) to their corresponding output values (y),
using only in-context examples. By focusing on inductive reasoning and
separating it from LLM-based deductive reasoning, we can isolate and
investigate inductive reasoning of LLMs in its pure form via SolverLearner. Our
observations reveal that LLMs demonstrate remarkable inductive reasoning
capabilities through SolverLearner, achieving near-perfect performance with ACC
of 1 in most cases. Surprisingly, despite their strong inductive reasoning
abilities, LLMs tend to relatively lack deductive reasoning capabilities,
particularly in tasks involving “counterfactual” reasoning.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要