\documentclass[11pt]{article}

\usepackage{picinpar}
\usepackage{subfigure}
\usepackage{url}
\usepackage{ifpdf}     % For pdf and postscript files

\usepackage{graphicx}        % Standard graphics package
\ifpdf
\DeclareGraphicsExtensions{.pdf,.png,.jpg,.tif}
\else
\DeclareGraphicsExtensions{.eps,.ps}
\fi

%=============================================================================
\setlength{\evensidemargin}{-0.0in}
\setlength{\oddsidemargin} {-0.0in}
\setlength{\textwidth}     {+6.50in}
\setlength{\topmargin}     {-0.5in}
\setlength{\textheight}    {+8.5in}
%=============================================================================
\parskip 0.07in            % Skip 0.05in between paragraphs
\setlength{\floatsep}{1\floatsep}         %inc. sp. between figures
\setlength{\textfloatsep}{1\textfloatsep} %inc. sp. bet. fig. on end and text
\setlength{\intextsep}{1\intextsep}       %inc. sp. bet. fig. in mid. and text

\begin{document} 
\pagestyle{empty}
\bibliographystyle{plain}
%\title
{\center
{\bf \Large TAO Applications: External Impact}

%\author
{\large
Steven J. Benson, Lois Curfman McInnes, Jorge J. Mor\'e, Jason Sarich\\
Mathematics and Computer Science Division \\
Argonne National Laboratory \\
Argonne, IL, 60439\\
}
%\date{March 19, 2002}
% \date{\today} 
}

\vspace{0.5in}

\nocite{*}

\noindent
\textbf{Parallel restricted maximum likelihood estimation for linear 
    models}    
\cite{malard02parallel}

\noindent
Restricted maximum likelihood (REML) estimation of variance
covariance matrices is an optimization problem that has both
scientific and industrial applications. Parallel REML
gradient algorithms are presented and compared for linear
models whose covariance matrix is large, sparse and possibly
unstructured. These algorithms are implemented using
publicly available toolkits and demonstrate that REML
estimates of large, sparse covariance matrices can be
computed efficiently on multicomputers with hundreds of
processors by using an effective mixture mixture of data
distributions together with a mixture of dense and sparse
linear algebra kernels.


\bigskip
\noindent
\textbf{Scalable parallel micromagnetic solvers for magnetic
             nanostructures} 
\cite{scholz03scalable}

\noindent
 A parallel finite element micromagnetics package
has been implemented, that is highly scalable, easily
portable and combines different solvers for the micromagnetic
equations. The implementation is based on the standard
Galerkin discretization on tetrahedral meshes with linear
basis functions. A static energy minimization, a dynamic
time integration, and the nudged elastic band method have
been implemented. The details of the implementation and some
aspects of the optimization are discussed and timing and
speedup results are given. Nucleation and magnetization
reversal processes in permalloy nanodots are investigated
with this micromagnetics package.

\bigskip
\noindent
\textbf{Nonlinear Conjugate Gradient Methods and Their Implementations
  by TAO on Dawning 2000-II} 
\cite{wiang03cgmethods}

\noindent
Nonlinear conjugate gradient (CG) method is a typical unconstrained
optimization method. TAO has three implementations of CG method:
CG\_FR, CG\_PR and CG\_PRP. In this paper, we describe their
implementations in TAO and give a test result.


\bigskip
\noindent
\textbf{Evaluation and Extension of Maximum Entropy Models with
  Inequality Constraints} 
\cite{kazama03evaluation}

\noindent
A maximum entropy (ME) model is usually estimated so that it conforms
to equality constraints on feature expectations. However, the
equality constraint is inappropriate for sparse and therefore
unreliable features. This study explores an ME model with box-type
inequality constraints, where the equality can be violated to
reflect this unreliability. We evaluate the inequality ME model
using text categorization datasets. We also propose an extension of
the inequality ME model, which results in a natural integration with
the Gaussian MAP estimation. Experimental results demonstrate the
advantage of the inequality models and the proposed extension.


\bigskip
\noindent
\textbf{A comparison of algorithms for maximum entropy parameter
  estimation}
\cite{malouf02comparison}

\noindent
Conditional maximum entropy (ME) models provide a general purpose
machine learning technique which has been successfully applied to
fields as diverse as computer vision and econometrics, and which is
used for a wide variety of classification problems in natural language
processing. However, the flexibility of ME models is not without
cost. While parameter estimation for ME models is conceptually
straightforward, in practice ME models for typical natural language
tasks are very large, and may well contain many thousands of free
parameters. In this paper, we consider a number of algorithms for
estimating the parameters of ME models, including iterative scaling,
gradient ascent, conjugate gradient, and variable metric
methods. Surprisingly, the standardly used iterative scaling
algorithms perform quite poorly in comparison to the others, and for
all of the test problems, a limited-memory variable metric algorithm
outperformed the other choices.


\bigskip
\noindent
\textbf{Algorithms for Linguistic Processing. NWO PIONIER Progress
  Report.}
\cite{Noord02algorithms}

\noindent
The proposal Algorithms for Linguistic Processing focuses on two
crucial problem areas in computational linguistics: problems of
processing efficiency and ambiguity. For the problem of efficiency
grammar approximation techniques will be investigated, whereas a
number of grammar specialization techniques are proposed for the
ambiguity problem.


\bigskip
\noindent
\textbf{Parallel restricted maximum likelihood estimation for linear 
    models with a dense exogenous matrix}
\cite{malard02parallel}

\noindent
Maximum likelihood estimates of covariance matrices for linear models
occur in many statistical and stochastic applications such as
estimating the genetic potential of cattle, financial time series
analysis, the characterization of chemical mixtures and in general in
the extimation of the parameters for stochastic differential
equations.  Restricted Maximum Likelihood (REML) is widely used in
application areas where sampling bias is an important concern, but
REML estimates are expensive to compute.  Parallel implementations
solely based on parallel dense matrix kernels need not scale well.
This paper demonstrates that it is possible to compute estimates of
covariance matrix for linear models based on restricted maximum
likelihood (REML) efficiently on parallel computers.  Two approaches
to computing in parallel the gradient of the REML objective function
are presented and compared.  The covariance matrix is not assumed
block diagonal.  The implementations presented are based on PETSc and
can run on any parallel computer supporting MPI.

\bigskip
\noindent
\textbf{Efficient Training of Conditional Random Fields}
\cite{wallach}

\noindent
This thesis explores a number of parameter estimation techniques for
conditional random  elds, a recently introduced [31] probabilistic
model for labelling and segmenting sequential data. Theoretical and
practical disadvantages of the training techniques reported in current
literature on CRFs are discussed. We hypothesise that general
numerical optimisation techniques result in improved performance over
iterative scaling algorithms for training CRFs. Experiments run on a a
subset of a well-known text chunking data set [28] con rm that this is
indeed the case. This is a highly promising result, indicating that
such parameter estimation techniques make CRFs a practical and ef
cient choice for labelling sequential data, as well as a theoretically
sound and principled probabilistic framework.



\bigskip
\noindent
\textbf{Client-Server Component Architecture for Scientific Computing}
\cite{dajani}

\noindent
In a Distributed Computing Environment software components dispersed
on a variety of computer platforms communicate transparently with each
other to emulate a single computer platform. One distributed component
framework model consists of two autonomous processes: the client and
the server. The client-server model implemented in an object-oriented
language shields low level platform complexities from the user and
allows coupling of prefabricated components. These components must
have a means of interfacing with each other in a distributed
environment. To accommodate this need, while maintaining high
performance, we propose a low level socket communication core. We
employ the proxy design pattern and introduce new C++ classes to
dynamically extend object behavior to a distributed environment. These
classes also serve as component interfaces. Here, we describe general
guidelines for partitioning objects into client and server components.

\bigskip
\noindent
\textbf{Mechanistic Process Modeling for Subsurface Remediation}
\cite{yabusaki02mechanistic}
\noindent
A key difficulty of simulating subsurface processes is the
need for bulk parameterizations to represent behaviors that integrate
over multiple length scales of spatially variable material
properties. PNNL is using NWMPP1 simulations of highly-resolved
depictions of micro-scale subsurface environments to determine these
effective parameters through a mechanistically based upscaling
procedure. In this case, the diffusive transport of reactive solutes
in biofilms was examined to determine the effective (upscaled)
diffusivity in biofilm systems as a function of (i) microscale
geometry, (ii) the microscale transport properties, and (iii) the
protein-mediated transmembrane transport kinetics. The importance of
this work is that it helps to explain what microscale factors are
important for describing mass transfer in biofilms and tissues.


\bigskip
\noindent
\textbf{Relevance Models to Help Estimate Document and Query Models}
\cite{bodoff04relevance}

\bigskip
\noindent
\textbf{Sourcebook of Parallel Computing}
\cite{crpchandbook}

\noindent
This book represents the collected knowledge and experience of over 60
leading parallel computing researchers. They offer students,
scientists and engineers a complete sourcebook with solid coverage of
parallel computing hardware, programming considerations, algorithms,
software and enabling technologies, as well as several parallel
application case studies. The Sourcebook of Parallel Computing offers
extensive tutorials and detailed documentation of the advanced
strategies produced by research over the last two decades
application case studies. The Sourcebook of Parallel Computing offers
extensive tutorials and detailed documentation of the advanced
strategies produced by research over the last two decades


\bigskip
\noindent
\textbf{Implementation of a high performance parallel finite element
  micromagnetics package}
\cite{scholz03implementation}

\noindent
A new high performance scalable parallel  nite element micromagnetics
package has been implemented. It includes solvers for static energy
minimization, time integration of the Landau-Lifshitz-Gilbert
equation, and the nudged elastic band method.


\bigskip
\noindent
\textbf{Optimization of a fed-batch fermentation process control 
competition problem using the NEOS server}
\cite{liang03optimization}

\noindent
An optimal control solution to a fed-batch fermentation process, 
responding to a competition call, was developed using NEOS Server. 
Substantial improvement to the nominal performance achieved in the 
paper demonstrates the ability of the NEOS Server and the APPS algorithm. 

\bigskip
\noindent
\textbf{Solving tough optimal control problems by Network Enabled Optimization Server (NEOS)}
\cite{liang03solving}

\noindent
The Network Enabled Optimization Server
(NEOS) for solving tough optimal control problems (OCPs) is
introduced. Though NEOS is generally used to solve static optimization
problems, in solving dynamic optimizations (OCPs)
in a general form, we show that the NEOS Server has some
advantages over the existing dedicated optimal control software
packages. After a step by step introduction of how to solve
a textbook bang-bang control problem by using NEOS, a
tough fed-batch fermentation process optimal control problem
is solved as a demonstration using NEOS with significantly
better performance.

\bigskip
\noindent
\textbf{Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking}
\cite{charniak-johnson:2005:ACL}

\noindent
Discriminative reranking is one method
for constructing high-performance statistical
parsers. A discriminative
reranker requires a source of candidate
parses for each sentence. This paper
describes a simple yet novel method
for constructing sets of 50-best parses
based on a coarse-to-fine generative parser. 
This method generates
50-best lists that are of substantially
higher quality than previously obtainable.
We used these parses as the input to a
MaxEnt reranker that selects the best
parse from the set of parses for each sentence,
obtaining an f-score of 91.0\% on
sentences of length 100 or less.


\bigskip
\noindent
\textbf{Discriminative language modeling with conditional random fields and the perceptron algorithm}
\cite{roark2004acl}

\noindent
This paper describes discriminative language modeling
for a large vocabulary speech recognition task. We contrast 
two parameter estimation methods: the perceptron
algorithm, and a method based on conditional random
fields (CRFs). The models are encoded as deterministic weighted 
finite state automata, and are applied by
intersecting the automata with word-lattices that are the
output from a baseline recognizer. The perceptron algorithm 
has the benefit of automatically selecting a relatively 
small feature set in just a couple of passes over the
training data. However, using the feature set output from
the perceptron algorithm (initialized with their weights),
CRF training provides an additional 0.5\% reduction in
word error rate, for a total 1.8\% absolute reduction from
the baseline of 39.2\%.

\bigskip
\noindent
\textbf{Raising the level of programming abstraction in scalable 
programming models}
\cite{bernholdt2004raising}

\noindent
The  complexity  of modern scientific  simulations
combined with  the  complexity  of the  high-performance
computer  hardware  on  which  they  run  place  an  
ever-increasing burden on scientific software developers, with
clear impacts on both productivity and performance.  We
argue  that  raising  the  level  of  abstraction  of  the
programming model/environment is a  key  element of
addressing this situation.  We present examples of two 
distinctly  different  approaches  to  raising  the  level  of 
abstraction of the programming model while maintaining
or  increasing performance: the  Tensor Contraction
engine,  a narrowly-focused  domain specific  language
together  with  an optimizing compiler;  and Extended 
Global Arrays, a programming framework that integrates
programming models dealing with different layers of the 
memory/storage hierarchy  using  compiler  analysis and
code transformation techniques. 


\bigskip
\noindent
\textbf{An interactive environment for supporting the transition from simulation to optimization}
\cite{bischof2003interactive}

\noindent
Numerical simulation is a powerful tool in science and engineering,
and it is also used for optimizing the design of products and
experiments rather than only for reproducing the behavior of
scientific and engineering systems. In order to reduce the number of
simulation runs, the traditional "trial and error" approach for
finding near-to-optimum design parameters is more and more replaced
with efficient numerical optimization algorithms. Done by hand, the
coupling of simulation and optimization software is tedious and
error-prone. In this note we introduce a software environment called
EFCOSS (Environment For Combining Optimization and Simulation
Software) that facilitates and speeds up this task by doing much of
the required work automatically. Our framework includes support for
automatic differentiation providing the derivatives required by many
optimization algorithms. We describe the process of integrating the
widely used computational fluid dynamics package FLUENT and a
MINPACK-1 least squares optimizer into EFCOSS and follow a sample
session solving a data assimilation problem.


\bigskip
\noindent
\textbf{Wide Coverage Parsing with Stochastic Attribute Value Grammars}
\cite{malouf2004wide}

\noindent
Stochastic Attribute Value Grammars
(SAVG) provide an attractive framework
for syntactic analysis, because they allow
the combination of linguistic sophistication
with a principled treatment of ambiguity.
The paper introduces a widecoverage
SAVG for Dutch, known as
Alpino, and we show how this SAVG can
be efficiently applied, using a beam search
algorithm to recover parses from a shared
parse forest. Experimental results for a
number of different corpora suggest that
the SAVG framework is applicable for realistically
sized grammars and corpora.





\bibliography{tao}

\end{document} 
