Seokhwan Kim
Staff Software Engineer, Google Cloud AI
Contact
1175 Borregas Ave, Sunnyvale, CA 94089, USA
+1 (650) 426-9701
Research Interests
Natural Language Processing
Spoken Dialogue Systems
Natural Language Understanding
Large Language Model
Work Experiences
Staff Software Engineer
Jun 2024 -
Cloud AI
Principal Applied Scientist
Oct 2021 - May 2024
Alexa AI
Senior Applied Scientist
Jun 2019 - Oct 2021
Alexa AI
NLP Research Scientist
Jul 2017 - May 2019
Creative Intelligence Lab.
Research Scientist
Jan 2012 - Jul 2017
Dialogue Technology Lab.,
Human Language Technology Department
Human Language Technology Department
Research Intern
Apr 2011 - Aug 2011
Databases and Information Systems
Advisor: Prof. Gerhard Weikum
Advisor: Prof. Gerhard Weikum
Education
Ph.D.
Sep 2005 - Feb 2012
Computer Science and Engineering
Dissertation: Cross-Lingual Weakly-Supervised Learning of Semantic Relations
Advisor: Gary Geunbae Lee
Advisor: Gary Geunbae Lee
B.S.
Mar 2001 - Aug 2005
Computer Science and Engineering
Professional Experiences
Virtual Infrastructure Co-Chair
2022
Challenge & Demonstration Co-Chair
2022
Panelist
2022
Speech and Language Processing Technical Committee (SLTC)
2021-2023
Board Member
2021-2023
Challenge Chair
2019
Organizing Committee
2017-2020
Organizing Committee
2015-2016
Local Organizing Committee Co-chair
2013
Publicity Committee Member
2012
Area Chair
Reviewer
AAAI, ACL, ACM-SIGIR, AI Magazine, CSL, CoNLL, EACL, EMNLP, ICLR, IEEE ASRU, IEEE ICASSP, IEEE SLT, IEEE/ACM TASLP, IJCAI, IJCNLP, IWSDS, Interspeech, NAACL-HLT, NeurIPS
Tutorials and Invited Talks
2023
"How Robust R U?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations.
Invited Talk @ Hugging Face Audio Transformers Course Launch Event, 14 Jun 2023.
2020
Summary of the Eighth Dialog System Technology Challenge (DSTC8).
Invited Talk @ The Third Workshop on Reasoning and Learning for Human-Machine Dialogues (DEEP-DIAL 2020), 08 Feb 2020.
Just ask: An Interactive Learning Framework for Vision and Language Navigation.
Invited Talk @ AI Assistant Summit (RE•WORK), 30 Jan 2020.
Publications
2023
[C70]
CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs.
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023), Singapore, Dec 2023.
[C69]
"What do others think?": Task-Oriented Conversational Modeling with Subjective Knowledge.
Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2023), Prague, Sep 2023.
[C68]
Investigating the Representation of Open Domain Dialogue Context for Transformer Models.
Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2023), Prague, Sep 2023.
[J10]
Overview of the Tenth Dialog System Technology Challenge: DSTC10.
IEEE/ACM Transactions on Audio, Speech, and Language Processing, Jul 2023.
[C67]
Identifying Entrainment in Task-oriented Conversations.
Proceedings of the 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023), Rhodes Island, Jun 2023.
[C66]
PLACES: Prompting Language Models for Social Conversation Synthesis.
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023), Dubrovnik, May 2023. (Findings)
[C65]
Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information.
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023), Dubrovnik, May 2023.
2022
[C64]
Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networks.
Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2022), Edinburgh, Sep 2022.
[J9]
Towards Textual Out-of-Domain Detection Without In-Domain Labels.
IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30 (1386-1395), Mar 2022.
2021
[C61]
Towards Zero and Few-shot Knowledge-seeking Turn Detection in Task-orientated Dialogue Systems. (Best Paper)
Proceedings of the Efficient Natural Language and Speech Processing NeurIPS Workshop (ENLSP 2021), Virtual, Dec 2021.
[C60]
"How robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations.
Proceedings of the 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU 2021), Virtual, Dec 2021.
[C59]
Rome was built in 1776: A case study on factual correctness in knowledge-grounded response generation.
Proceedings of the 3rd Workshop on NLP for Conversational AI (NLP4ConvAI), Punta Cana, Nov 2021.
[J8]
Editorial: Special Issue on the Eighth Dialog System Technology Challenge.
IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29 (2434-2436), Aug 2021.
[C57]
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2021), Singapore, Jul 2021.
[C56]
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2021), Singapore, Jul 2021.
[J7]
Overview of the Eighth Dialog System Technology Challenge: DSTC8.
IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29 (2529-2540), May 2021.
[C55]
Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access Track in DSTC9.
Proceedings of the AAAI-21 Workshop on Dialog System Technology Challenges (DSTC9), Virtual, Feb 2021.
2020
[C54]
Policy-Driven Neural Response Generation for Knowledge-Grounded Dialog Systems.
Proceedings of the 13th International Conference on Natural Language Generation (INLG 2020), Dublin, Dec 2020.
[C53]
Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access.
Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2020), pp. 278-289, Boise, Jul 2020.
[C52]
Proceedings of the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence (IJCAI-PRICAI 2020), pp. 1061-1068, Yokohama, Jul 2020.
(12.6% acceptance)
[C49]
Just ask: An Interactive Learning Framework for Vision and Language Navigation.
Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), New York, Feb 2020.
(20.6% acceptance)
2019
[C48]
The Eighth Dialog System Technology Challenge.
Proceedings of the 3rd Conversational AI workshop: Today's Practice and Tomorrow's Potential (ConvAI 2019), Vancouver, Dec 2019.
[C47]
Analyzing Sentence Fusion in Abstractive Summarization.
Proceedings of the EMNLP 2019 Workshop on New Frontiers in Summarization (NewSum), Hong Kong, Nov 2019.
[C44]
Deep Recurrent Neural Networks with Layer-wise Multi-head Attentions for Punctuation Restoration.
Proceedings of the 44th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2019), pp. 7280-7284, Brighton, May 2019.
[J6]
Overview of the Sixth Dialog System Technology Challenge: DSTC6.
Computer Speech and Language, 55 (1-25), May 2019.
2018
[C42]
Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018), pp. 615-621, New Orleans, Jun 2018.
(29.4% acceptance)
2017
[C40]
Truly Multi-modal YouTube-8M Video Classification with Video, Audio, and Text (Team DL2.0, ranked at 22/650).
Proceedings of the CVPR 2017 Workshop on YouTube-8M Large-Scale Video Understanding (YouTube-8M), Honolulu, Jul 2017.
[J5]
Neural sentence embedding using only in-domain sentences for out-of-domain sentence detection in dialog systems.
Pattern Recognition Letters, 88 (26-32), Mar 2017.
2016
[C37]
Proceedings of the 7th International Workshop on Spoken Dialogue Systems (IWSDS 2016), Saariselkä, Jan 2016.
2015
[C36]
A Robust Spoken Q&A System with Scarce In-Domain Resources.
Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2015 (APSIPA ASC 2015), Hong Kong, Dec 2015.
[C35]
Configuration of Dialogue Agent with Multiple Knowledge Sources.
Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2015 (APSIPA ASC 2015), Hong Kong, Dec 2015.
[C34]
Towards Improving the Performance of Vector Space Model for Chinese Frequently Asked Question.
Proceedings of the 19th International Conference on Asian Language Processing (IALP 2015), Suzhou, Oct 2015.
[C33]
Conversational Agent and Management Tools for Conference and Tourism Domain.
Proceedings of the 16th Annual Conference of the International Speech Communication Association (Interspeech 2015), Dresden, Sep 2015. (Show and Tell)
[C32]
Towards Improving Dialogue Topic Tracking Performances with Wikification of Concept Mentions.
Proceedings of the 16th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL 2015), pp. 124-128, Prague, Sep 2015.
[C31]
Wikification of Concept Mentions within Spoken Dialogues Using Domain Constraints from Wikipedia.
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 2225-2229, Lisbon, Sep 2015.
(24.0% acceptance)
[C30]
CLARA: a multifunctional virtual agent for conference support and touristic information.
Proceedings of the 6th International Workshop on Spoken Dialog System (IWSDS 2015), Busan, Jan 2015. (Demo)
2014
[C29]
Design and Evaluation of a Conversational Agent for the Touristic Domain.
Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2014 (APSIPA ASC 2014), Siem Reap, Dec 2014.
[C28]
An empirical evaluation of an IR-based strategy for chat-oriented dialogue systems.
Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2014 (APSIPA ASC 2014), Siem Reap, Dec 2014.
[C27]
R-cube: a dialogue agent for Restaurant Recommendation and Reservation.
Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference 2014 (APSIPA ASC 2014), Siem Reap, Dec 2014.
[C25]
SARA - Singapore's Automated Responsive Assistant for the Touristic Domain.
Proceedings of the 15th Annual Conference of the International Speech Communication Association (Interspeech 2014), pp. 2138-2139, Singapore, Sep 2014. (Show and Tell)
[C24]
Spoken Dialogue System for Restaurant Recommendation and Reservation.
Proceedings of the 15th Annual Conference of the International Speech Communication Association (Interspeech 2014), pp. 1488-1489, Singapore, Sep 2014. (Show and Tell)
[C23]
SARA - Singapore's Automated Responsive Assistant - A Multimodal Dialogue System for Touristic Information.
Proceedings of the 11th International Conference on Mobile Web Information Systems (MobiWIS 2014), pp. 153-164, Barcelona, Aug 2014.
[C22]
Proceedings of the 15th Annual SIGdial Meeting on Discourse and Dialogue (SIGDIAL 2014), pp. 332-336, Philadelphia, Jun 2014.
[C20]
Proceedings of the 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2014), pp. 131-135, Florence, May 2014.
[C19]
AIDA: An Avatar-Supported Multimodal Dialogue System.
Proceedings of the 27th International Conference on Computer Animation and Social Agents (CASA 2014), Houston, May 2014.
[J4]
Cross-Lingual Annotation Projection for Weakly-Supervised Relation Extraction.
ACM Transactions on Asian Language Information Processing, 13:1 (3), Feb 2014.
[C18]
Web-based Multimodal Multi-domain Spoken Dialogue System.
Proceedings of the 5th International Workshop on Spoken Dialog Systems (IWSDS 2014), Napa, Jan 2014.
2013
[C17]
Proceedings of the 14th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2013), pp. 145-147, Metz, Aug 2013. (Demo)
[C16]
A Graph-based Cross-lingual Projection Approach for Spoken Language Understanding Portability to a New Language.
Proceedings of the 38th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013), pp. 8332-8336, Vancouver, May 2013.
2012
[C15]
A Two-Step Approach for Efficient Domain Selection in Multi-Domain Dialog Systems.
Proceedings of the 4th International Workshop on Spoken Dialog System (IWSDS 2012), pp. 105-111, Ermenonville, Nov 2012.
[C12]
Seamless error correction interface for voice word processor.
Proceedings of the 37th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), pp. 4973-4976, Kyoto, Mar 2012.
2011
[C11]
Multi-domain spoken dialog system for information access in mobile environment.
Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (IEEE ASRU 2011), Hawaii, Dec 2011. (Demo)
[C10]
A Cross-lingual Annotation Projection-based Self-supervision Approach for Open Information Extraction.
Proceedings of the 5th international joint conference on natural language processing (IJCNLP 2011), pp. 741-748, Chiang Mai, Nov 2011.
(27.0% acceptance)
[C9]
Web-search enhanced contents retrieval for information access dialog system.
Proceedings of the 12th Annual Conference of the International Speech Communication Association (Interspeech 2011), pp. 1297-1300, Florence, Aug 2011.
[J3]
A Local Tree Alignment Approach to Relation Extraction of Multiple Arguments.
Information Processing and Management, 47:4 (593-605), Jul 2011.
2010
[C8]
Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pp. 564-571, Beijing, Aug 2010.
(19.0% acceptance)
2009
[C7]
A Local Tree Alignment-based Soft Pattern Matching Approach for Information Extraction.
Proceedings of the North American Chapter of the Association for Computational Linguistics/Human Language Technology (NAACL HLT 2009), pp. 169-172, Colorado, May 2009.
[J2]
Example-based dialog modeling for practical multi-domain dialog system.
Speech Communications, 51:5 (466-484), May 2009.
2008
[J1]
DialogStudio: A workbench for data-driven spoken dialog system development and management.
Speech Communications, 50:8-9 (683-697), , Aug-Sep 2008.
[C5]
An alignment-based approach to semi-supervised relation extraction including multiple arguments.
Proceedings of the fourth Asian Information Retrieval Symposium (AIRS 2008), Harbin, Jan 2008.
2007
[C4]
Example-based spoken dialog processing for guidance robots.
Proceedings of the 4th international conference on ubiquitous robots and ambient intelligence (URAI 2007), Pohang, Nov 2007.
[C3]
Proceedings of The 16th IEEE International Symposium on Robot and Human Interactive Communication (IEEE RO-MAN 2007), pp. 178-181, Jeju, Aug 2007.
[C2]
A semi-supervised method for efficient construction of statistical spoken language understanding resources.
Proceedings of the 8th Annual Conference of the International Speech Communication Association (Interspeech 2007), pp. 2797-2800, Antwerp, Aug 2007.
2006
[C1]
Proceedings of the Human Language Technology/North American Chapter of the Association for Computational Linguistics (HLT-NAACL 2006), pp. 69-72, New York, Jun 2006.
Patents
[P18]
Answering questions during video playback.
Registration #US11544590B2, USA, 03 Jan 2023.
[P17]
Generating a response to a user query utilizing visual features of a video segment and a query-response-neural network.
Registration #US11244167B2, USA, 08 Feb 2022.
[P16]
Automatic text segmentation based on relevant context.
Registration #US11210470B2, USA, 28 Dec 2021.
[P15]
Generating ground truth annotations corresponding to digital image editing dialogues for training state tracking models.
Registration #US11100917B2, USA, 24 Aug 2021.
[P14]
Utilizing a dynamic memory network to track digital dialog states and generate responses.
Registration #US10909970B2, USA, 02 Feb 2021.
[P13]
Emphasizing key points in a speech file and structuring an associated transcription.
Registration #US10783314B2, USA, 22 Sep 2020.
[P12]
Entity Name Tagging Method Capable of Applying an Unsupervised Learning Method Including a Restriction Condition and a Device Thereof.
Registration #10-1255957, South Korea, 11 Apr 2013.
[P11]
Conversation Classification Method Capable of Classifying Conversation Intention by Using a Hidden Markov Model.
Registration #10-1255468, South Korea, 10 Apr 2013.
[P10]
Information Search Method by Using the Web Capable of Using a Vector Space Database and a Voice Converstaion Method Using the Method.
Registration #10-1252397, South Korea, 02 Apr 2013.
[P9]
Speech Processing Device Capable of Effective Input Modification and a Method Thereof.
Registration #10-1197010, South Korea, 26 Oct 2012.
[P8]
Korean Open Type Information Extracting Method and Program Readable Recording Medium Capable of Increasing Korean Open Type Information Extracting Performances.
Registration #10-1180589, South Korea, 31 Aug 2012.
[P7]
Method for searching for information using the web and method for voice conversation using same.
Registration #PCT/KR2012/004405, 28 Aug 2012.
[P6]
User profile automatic creation apparatus through voice dialog meaning process, and contents recommendation apparatus using the same.
Registration #10-1072176, South Korea, 04 Oct 2011.
[P5]
Method and apparatus for searching voice data from audio and video data under the circumstances including unregistered words.
Registration #10-1069534, South Korea, 26 Sep 2011.
[P4]
Method of automatically detecting dangerous situation using natural language extraction and apparatus thereof.
Registration #10-1023031, South Korea, 10 Mar 2011.
[P3]
Method and apparatus for automatically constructing ontology from non-structure web documents.
Registration #10-0917176, South Korea, 07 Sep 2009.
[P2]
Method and system for recognizing biological named entity based on workbench.
Registration #10-0825687, South Korea, 22 Apr 2008.
[P1]
Dialog management apparatus and method for chatting agent.
Registration #10-0818979, South Korea, 27 Mar 2008.