An Essay on Optimization Mystery of Deep Learning
Despite the huge empirical success of deep learning, theoretical understanding of neural networks learning process is still lacking. This is the reason, why some of its features seem "mysterious". We emphasize two mysteries of deep learning: generalization mystery, and optimization mystery. In this essay we review and draw connections between several selected works concerning the latter.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningSimilar Papers 제목 키워드 기반
Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring
Multi-trait essay scoring aims to provide fine-grained evaluation of writing quality across multiple dimensions. However, how to effectively post-train autoregressive scoring models remains underexplored. In this paper, …
Towards Robust Argumentative Essay Understanding via TIDE: An Interactive Framework with Trial and Debate
Argumentative essays serve as a vital medium for assessing critical thinking and reasoning skills, yet there is limited works on accurately understanding and evaluating such texts via prompt. In this work, we propose TID…
Automated Essay ScoringExploring Optimal Voting in Native Language Identification
We describe the submissions entered by the National Research Council Canada in the NLI-2017 evaluation. We mainly explored the use of voting, and various ways to optimize the choice and number of voting systems. We also …
Language IdentificationNative Language IdentificationWho Killed Albert Einstein? From Open Data to Murder Mystery Games
This paper presents a framework for generating adventure games from open data. Focusing on the murder mystery type of adventure games, the generator is able to transform open data from Wikipedia articles, OpenStreetMap a…
ArticlesDecision-Level Ordinal Modeling for Multimodal Essay Scoring with Large Language Models
Automated essay scoring (AES) predicts multiple rubric-defined trait scores for each essay, where each trait follows an ordered discrete rating scale. Most LLM-based AES methods cast scoring as autoregressive token gener…
Automated Essay Scoring