Publications

Instruction-guided image-to-image (I2I) editors are increasingly entering consumer and professional visual workflows, where trustworthiness depends not only on prompt compliance but also on equitable preservation of identity-relevant attributes. We formalize two failure modes Soft Erasure, where requested edits are weakly realized or silently suppressed, and Stereotype Replacement, where edits introduce unrequested, stereotype-consistent demographic attributes and evaluate them across three recent open-weight editors on 5,040 edited portraits. We find that 62-71% of outputs exhibit skin lightening, with Indian and Black source portraits affected at 72-75% compared with 44\% for White source portraits, a pattern consistent with output-level drift toward lighter or more White-presenting appearances when identity constraints are underspecified. In a mitigation case study, prompt-level appearance constraints reduce race-change scores for non-White source portraits by up to 1.48 points, with negligible change for White source portraits, without modifying model weights. Together, these findings show that identity preservation is not a uniform property of I2I portrait editing systems, but an unevenly distributed trustworthiness failure with direct social consequences. At deployment scale, such silent distortions can shape AI-mediated self-representation and reinforce representational disparities. We introduce a controlled audit protocol for fairness-aware evaluation and governance of generative editing systems.

Large vision-language models (LVLMs) struggle to reliably detect visual primitives in charts and align them with semantic representations, which severely limits their performance on complex visual reasoning. This lack of perceptual grounding constitutes a major bottleneck for chart-based reasoning. We propose VisDoT, a framework that enhances visual reasoning through human-like interpretation grounding. We formalize four perceptual tasks based on the theory of graphical perception, including position and length. Building on this foundation, we introduce Decomposition-of-Thought (DoT) prompting, which sequentially separates questions into visual perception sub-questions and logic sub-questions. Fine-tuning InternVL with VisDoT achieves a +11.2% improvement on ChartQA and surpasses GPT-4o on the more challenging ChartQAPro benchmark. On the newly introduced VisDoTQA benchmark, the model improves by +33.2%. Furthermore, consistent zero-shot gains on diverse open-domain VQA benchmarks confirm the generalizability of the perception-logic separation strategy for visual question answering. VisDoT leverages human-like perception to enhance visual grounding, achieving state-of-the-art chart understanding and interpretable visual reasoning.

Generative image models produce striking visuals yet often misrepresent culture. Prior work has examined cultural bias mainly in text-to-image (T2I) systems, leaving image-to-image (I2I) editors underexplored. We bridge this gap with a unified evaluation across six countries, an 8-category/36-subcategory schema, and era-aware prompts, auditing both T2I generation and I2I editing under a standardized protocol that yields comparable diagnostics. Using open models with fixed settings, we derive cross-country, cross-era, and cross-category evaluations. Our framework combines standard automatic metrics, a culture-aware retrieval-augmented VQA, and expert human judgments collected from native reviewers. To enable reproducibility, we release the complete image corpus, prompts, and configurations. Our study reveals three findings: (1) under country-agnostic prompts, models default to Global-North, modern-leaning depictions that flatten cross-country distinctions; (2) iterative I2I editing erodes cultural fidelity even when conventional metrics remain flat or improve; and (3) I2I models apply superficial cues (palette shifts, generic props) rather than era-consistent, context-aware changes, often retaining source identity for Global-South targets. These results highlight that culture-sensitive edits remain unreliable in current systems. By releasing standardized data, prompts, and human evaluation protocols, we provide a reproducible, culture-centered benchmark for diagnosing and tracking cultural bias in generative image models.

Social norms govern culturally appropriate behavior in communication, enabling dialogue systems to produce responses that are not only coherent but also socially acceptable. We present NormGenesis, a multicultural framework for generating and annotating socially grounded dialogues across English, Chinese, and Korean. To model the dynamics of social interaction beyond static norm classification, we propose a novel dialogue type, Violation-to-Resolution (V2R), which models the progression of conversations following norm violations through recognition and socially appropriate repair. To improve pragmatic consistency in underrepresented languages, we implement an exemplar-based iterative refinement early in the dialogue synthesis process. This design introduces alignment with linguistic, emotional, and sociocultural expectations before full dialogue generation begins. Using this framework, we construct a dataset of 10,800 multi-turn dialogues annotated at the turn level for norm adherence, speaker intent, and emotional response. Human and LLM-based evaluations demonstrate that NormGenesis significantly outperforms existing datasets in refinement quality, dialogue naturalness, and generalization performance. We show that models trained on our V2R-augmented data exhibit improved pragmatic competence in ethically sensitive contexts. Our work establishes a new benchmark for culturally adaptive dialogue modeling and provides a scalable methodology for norm-aware generation across linguistically and culturally diverse languages.

Prompt design is a primary control interface for large language models (LLMs), yet standard evaluations largely reduce performance to answer correctness, obscuring why a prompt succeeds or fails and providing little actionable guidance. We propose PEEM (Prompt Engineering Evaluation Metrics), a unified framework for joint and interpretable evaluation of both prompts and responses. PEEM defines a structured rubric with 9 axes: 3 prompt criteria (clarity/structure, linguistic quality, fairness) and 6 response criteria (accuracy, coherence, relevance, objectivity, clarity, conciseness), and uses an LLM-based evaluator to output (i) scalar scores on a 1-5 Likert scale and (ii) criterion-specific natural-language rationales grounded in the rubric. Across 7 benchmarks and 5 task models, PEEM's accuracy axis strongly aligns with conventional accuracy while preserving model rankings (aggregate Spearman rho about 0.97, Pearson r about 0.94, p < 0.001). A multi-evaluator study with four models shows consistent relative judgments (pairwise rho = 0.68-0.85), supporting evaluator-agnostic deployment. Beyond alignment, PEEM captures complementary linguistic failure modes and remains informative under prompt perturbations: prompt-quality trends track downstream accuracy under iterative rewrites, semantic adversarial manipulations induce clear score degradation, and meaning-preserving paraphrases yield high stability (robustness rate about 76.7-80.6%). Finally, using only PEEM scores and rationales as feedback, a zero-shot prompt rewriting loop improves downstream accuracy by up to 11.7 points, outperforming supervised and RL-based prompt-optimization baselines. Overall, PEEM provides a reproducible, criterion-driven protocol that links prompt formulation to response behavior and enables systematic diagnosis and optimization of LLM interactions.

표 일부분에 대한 해석은 대규모 언어 모델이 표의 내용을 인식할 수 있는 형태로 구성하여 해석을 생성 하는 태스크다. 기존 연구들은 태스크에 맞는 데이터를 구축하거나 고성능의 LLM 모델을 미세 조정함으 로써, 상황에 맞는 태스크 해결에 초점을 두고 있다. 하지만 파라미터 수가 많아진 LLM 모델을 미세 조정 하기에 많은 자원과 시간이 소요된다. 따라서 한정된 자원으로도 유사한 결과를 낼 수 있도록 기존의 작 은 파라미터를 갖는 모델을 통해 텍스트 생성에 기초적인 틀을 마련하고, LLM 모델의 문장 수정 방식을 통한 2단계 추론 방식을 제안한다. 이 방식을 통해 많은 자원을 이용하여 LLM모델의 미세 조정할 필요없 이, 빠르고 효율적으로 표 일부에 대한 추론을 할 수 있다.

이미지 캡셔닝은 이미지를 설명하는 문장을 자동으로 생성하는 작업으로, 시각 장애인 지원, 의료 이미지 설명, 비디오 자막 생성 등에서 다양하게 사용된다. 기존 연구들은 다양한 모델을 사용하여 이미지 캡셔닝 작업을 수행해 왔으며 OCR 정보와 같은 추가적인 정보 추출을 통해 이미지를 더 잘 설명하는 캡션을 생 성하고자 하였다. 하지만 영어권 이외의 이미지 캡션 생성 능력은 떨어지는 편이다. 따라서 한국어 이미지 캡션 생성의 성능을 높이고자 OCR 정보를 CoT 프롬프트와 결합하여 최종 캡션을 생성하는 방법을 제안 한다. 이 방법을 통해 기존의 방식에 비교하여 성능 향상을 얻을 수 있으며, 특히 불필요한 정보를 생성하 는 비율을 줄이는 효과가 있다.

우리 사회가 고령화시대로 접어들면서 낙상은 매우 심각한 사회문제가 되고 있으며 정확한 낙 상 감지 기술의 수요도 늘고 있다. 본 연구는 웹 캠을 이용한 개선된 낙상감지 기법을 제안한다. 제 안하는 기법은 RGB 영상을 기반으로 스켈레톤 포즈 추출, 데이터 가공, GRU(Gated Recurrent Unit) 신 경망 알고리즘을 적용한 낙상 감지 실험 및 감지 결과 분석의 과정이 포함된다.