Inferential-Strategies

홈페이지 · 논문 1편

A collection of large languge model responses to tasks of propositional logic. The responses are annotated according to the following criteria: - Inferential strategy employed by the model. Strategies considered are: *supposition following*, *chain construction*, *compound strategy*, *concatenation strategy*, and the *symbolic strategy*. Binary labels are assigned to each strategy, indicating whether the strategy is present in the model's response. - Assessment of the validity of the model's final conclusion. Binary labels are assigned to each response, indicating whether the model's conclusion is accurate ("valid_conclusion"). - Evaluation of the soundness of the model's rationale. Binary labels are assigned to each response, indicating whether the rationale provided by the model is sound ("sound_reasoning"). - A description of the model's reasoning error. This information is provided in the form of a string ("reasoning_errors").