MICHAEL J JOHNSTON email: johnston@research.att.com 180 PARK AVE - BUILDING 103 FLORHAM PARK, NJ, 07932

Interests

• Multimodal human-computer interaction
  • Integration and understanding
  • Dialog managment
  • Output generation
• Natural Language Processing
• Machine learning
• User interface design and evaluation
• Search and multimedia visualization
• Spoken dialogue systems
• Content extraction and distillation
• Mobile computing
• Pen and gesture-based interfaces
• Visual languages
• Information access in urban environments
• Medical informatics
• Web standards

Vita

• AT&T Labs Research, Florham Park, NJ
  
  • Senior Technical Specialist, 2004-Present
  • Principal Technical Staff, 1999-2004.
• Oregon Graduate Institute, Department of Computer Science and Engineering, Portland, OR
  
  • Assistant Professor, 1998-1999.
  • Principal Project Scientist, 1996-1997
• Brandeis University, Department of Computer Science, Waltham, MA
  
  • Post-doctoral Research Fellow, 1994-1996.
• Apple Computer Inc., Cupertino, CA
  
  • Summer Intern, 1993, 1994.
• Hewlett Packard Labs, Palo Alto, CA
  
  • Member of Technical Staff, 1990, 1992.

Publications

Johnston, M, P. Ehlen, D. Gibbon, Z. Liu. 2007. The Multimodal Presentation Dashboard. Proceedings of the NAACL-HLT 2007 Workshop: Bridging the Gap: Academic and Industrial Research in Dialog Technologies. pp. 17-24.

Johnston, M., L-F. D'Haro, M. Levine, B. Renger. 2007. A Multimodal Interface for Access to Content in the Home. Proceedings of the Association for Computational Linguistics 45th Annual Conference. pp. 376-383.

Johnston, M. and S. Bangalore. 2006. "Learning Edit Machines for Robust Multimodal Understanding".  IEEE International Conference on Acoustics, Speech, and Signal Processing.

Johnston, M. and S. Bangalore. 2005. "Finite-state Multimodal Integration and Understanding". Journal of Natural Language Engineering 11.2, pp. 159-187, Cambridge University Press.

Johnston, M. and S. Bangalore. 2005. "Combining Stochastic and Grammar-based Language Processing with Finite-state Edit Machines". In Proceedings of IEEE Automatic Speech Recognition and Understanding Workshop.

Johnston, M., W. Chou, D. Dahl, G. McCobb, D. Raggett. 2005. "EMMA: Extensible Multimodal Annotation Markup Language". W3C Last Call Working Draft. 

Johnston, M. and S. Bangalore. 2004. "MATCHKiosk: A Multimodal Interactive City Guide". In Proceedings of the ACL-2004 Interactive Posters/Demonstrations Session, pp. 222-225.

Bangalore, S. and M. Johnston. 2004. "Balancing Data-driven and Rule-based Approaches in the Context of a Multimodal Conversational System". In Proceedings of Human Language Technology conference / North American chapter of the Association for Computational Linguistics annual meeting (HLT-NAACL 2004).

Bangalore, S. and M. Johnston. 2004. "Robust Multimodal Understanding". Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP-2004).

Walker, M., S. Whittaker, A. Stent, P. Maloor, J. Moore, M. Johnston, G. Vasireddy. 2003. "Generation and Evaluation of User Tailored Responses in Multimodal Dialogue". Cognitive Science, Vol 28., Issue 5, September-October 2003, Rumelhart Prize Special Issue Honoring Aravind K. Joshi, pp. 811-840.

Johnston, M, S. Bangalore, G. Vasireddy, A. Stent, P. Ehlen, M. Walker, S. Whittaker, P. Maloor. 2002. "MATCH: An Architecture for Multimodal Dialogue Systems". In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.

Johnston, M. , S. Bangalore, A. Stent, G. Vasireddy, P. Ehlen. 2002. "Multimodal Language Processing for Mobile Information Access". In Proceedings of International Conference on Spoken Language Processing, Denver, Colorado.

Ehlen, P., G. Vasireddy, M. Johnston. 2002. "Collecting Mobile Multimodal Data for MATCH". In Proceedings of International Conference on Spoken Language Processing, Denver, Colorado.

Walker, M., S. Whittaker, A. Stent, P. Maloor, J. Moore, M. Johnston, G. Vasireddy. 2002. "Speech plans:generating evaluative responses in spoken dialogue". In Proceedings of International Natural Language Generation Conference.

Johnston, M., S. Bangalore, G. Vasireddy. 2001. "MATCH: Multimodal Access To City Help". In Proceedings of ASRU 2001 Workshop.  Madonna di Campiglio, Italy.

Johnston, M. and S. Bangalore. 2001. "Finite-state methods for multimodal parsing and integration". Finite-state Methods Workshop, ESSLLI Summer School on Logic Language and Information, Helsinki, Finland.

Johnston, M. and S. Bangalore. 2000. "Finite-state multimodal parsing and understanding". Proceedings of COLING-2000. Saarbruecken, Germany.

Johnston, M. 2000. "Deixis and conjunction in multimodal systems". In Proceedings of COLING-2000, Saarbruecken, Germany.

Bangalore, S. and M. Johnston. 2000. "Integrating Multimodal Language Processing with Speech Recognition". Proceedings of International Conference on Spoken Language Processing, Beijing, China.

Johnston, M. 1998. "Unification-based Multimodal Parsing". In Proceedings of the 17th International Conference on Computational Linguistics and 36^tthAnnual Meeting of the Association for Computational Linguistics. (COLING-ACL 98), pp. 624-630.

Cohen, P. R., M. Johnston, D. McGee, S. L. Oviatt, J. Clow, I. Smith. 1998. "The Efficiency of Multimodal Interaction: A Case Study". In Proceedings of the International Conference on Spoken Language Processing (ICSLP), Sydney, Australia.

Johnston, M., P. R. Cohen, D. McGee, S. L. Oviatt, J. Pittman, I. Smith. 1997. "Unification-based Multimodal Integration". Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics. Madrid, Spain.

Cohen, P. R., M. Johnston, D. McGee, S. L. Oviatt, J. Pittman, I. Smith, L. Chen, and J. Clow. 1997. "QuickSet: Multimodal interaction for distributed applications". In Proceedings of the Fifth ACM International Multimedia Conference. ACM Press: New York.

Cohen, P. R., M. Johnston, D. McGee, S. L. Oviatt, J. Pittman, I. Smith, L. Chen, and J. Clow. 1997. "Multimodal interaction for distributed interactive simulation". In Proceedings of Innovative Applications of Artificial Intelligence Conference. AAAI/MIT Press: Menlo Park.
 
Cohen, P. R., M. Johnston, D. McGee, I. Smith, S. L. Oviatt, J. Pittman, L. Chen, and J. Clow. 1997. "QuickSet: Multimodal interaction for simulation set-up and control". Proceedings of the Fifth Applied Natural Language Processing meeting. Association for Computational Linguistics: Washington, D.C.

Johnston, M. 1997. "The Telic/Atelic Distinction and the Individuation of Quantificational Domains".  in Empirical Issues in Formal Syntax and Semantics: Selected Papers from the Colloque de Syntaxe et de Sémantique de Paris (CSSP 95), Danièle Godard and Francis Corblin (Eds.). Peter Lang, Bern.

Cohen, P. R., L. Chen, J. Clow, M. Johnston, D. McGee, J. Pittman, and I. Smith. 1996. "Quickset: A multimodal interface for distributed interactive simulation". Proceedings of the UIST'96 Demonstration Session, Seattle.

Pustejovsky, J., B. Boguraev, M. Verhagen, P. Buitelaar, M. Johnston. 1997. "Semantic Indexing and Typed Hyperlinking". In Proceedings of the AAAI Spring Symposium on Natural Language Processing for the World Wide Web. American Association for Artificial Intelligence.

Johnston, M., B. Boguraev, and J. Pustejovsky.1995. "The Acquisition and Interpretation of Complex Nominals". In Working Notes of AAAI Spring Symposium on the Representation and Acquisition of Lexical Knowledge. American Association for Artificial Intelligence.

My Projects
	Multimodal Interfaces

My Patents
	7505908	Systems and methods for classifying and representing gestural inputs, 03-17-2009
	7499862	System and method for accessing and annotating electronic medical records using a multi-modal interface, 03-03-2009
	7451088	System and method of handling problematic input during context-sensitive help for multi-modal dialog systems, 11-11-2008
	7295975	Systems and methods for extracting meaning from multimodal inputs using finite-state devices, 11-13-2007
	7257575	Systems and methods for generating markup-language based expressions from multi-modal and unimodal inputs, 08-14-2007
	7225131	System and method for accessing and annotating electronic medical records using multi-modal interface, 05-29-2007
	7177815	System and method of context-sensitive help for multi-modal dialog systems, 02-13-2007
	7177816	System and method of handling problematic input during context-sensitive help for multi-modal dialog systems, 02-13-2007
	7069215	Systems and methods for extracting meaning from multimodal inputs using finite-state devices, 06-27-2006
	6868383	Systems and methods for extracting meaning from multimodal inputs using finite-state devices, 03-15-2005

Selected Publications
	MATCHkiosk: A Multimodal Interactive City Guide. Srinivas Bangalore, M. Johnston. Association of Computational Linguistics (ACL2004), Barcelona, Spain, 2004, 2004
	Robust Multimodal Understanding. Srinivas Bangalore, M. Johnston. ICASSP 2004, Montreal, Canada, 2004
	Balancing Data-Driven and Rule-Based Approaches in the Context Of A Multimodal Conversational System. Michael Johnston, . Proceedings of Human Language Technology Conference / North American Chapter of the Association for Computational Linguistics Annual Meeting (HLT-NAACL 2004), 2004
	Robust Multimodal Understanding. Michael Johnston, Srinivas Bangalore. Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP-2004), 2004
	Balancing Data-Driven And Rule-Based Approaches in the Context of a Multimodal Conversational System. Srinivas Bangalore, M. Johnston. ASRU: Automatic Speech Recognition and Understanding, 2003