Abstract
Automatic differentiation is a powerful technique for evalu- ating derivatives of functions given in the form of a high-level program- ming language such as Fortran, C, or C++. The program is treated as a potentially very long sequence of elementary statements to which the chain rule of differential calculus is applied over and over again. Combining automatic differentiation and the organizational structure of toolkits for parallel scientific computing provides a mechanism for eval- uating derivatives by exploiting mathematical insight on a higher level. In these toolkits, algorithmic structures such as BLAS-like operations, linear and nonlinear solvers, or integrators for ordinary differential equations can be identified by their standardized interfaces and recognized as high-level mathematical objects rather than as a sequence of elemen- tary statements. In this note, the differentiation of a linear solver with respect to some parameter vector is taken as an example. Mathematical insight is used to reformulate this problem into the solution of multiple linear systems that share the same coefficient matrix but differ in their right-hand sides. The experiments reported here use ADIC, a tool for the automatic differentiation of C programs, and PETSc, an object-oriented toolkit for the parallel solution of scientific problems modeled by partial differential equations.
This work was supported in part by the Mathematical, Information, and Computational Sciences Division subprogram of the Office of Advanced Scientific Computing Research, U.S. Department of Energy, under Contract W-31-109-Eng-38.
Chapter PDF
References
Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. PETSc 2.0 users manual. Technical Report ANL-95/11-Revision 2.0.24, Argonne National Laboratory, 1999.
Satish Balay, William D. Gropp, Lois Curfman McInnes, and Barry F. Smith. PETSc home page. http://www.mcs.anl.gov/petsc, 1999.
Martin Berz, Christian Bischof, George Corliss, and Andreas Griewank. Computational Differentiation: Techniques, Applications, and Tools. SIAM, Philadelphia, 1996.
Christian Bischof, Alan Carle, George Corliss, Andreas Griewank, and Paul Hovland. ADIFOR: Generating derivative codes from Fortran programs. Scientific Programming, 1(1):11–29, 1992.
Christian Bischof, Alan Carle, Peyvand Khademi, and Andrew Mauer. ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computational Science & Engineering, 3(3):18–32, 1996.
Christian Bischof, Lucas Roh, and Andrew Mauer. ADIC-An extensible automatic differentiation tool for ANSI-C. Software-Practice and Experience, 27(12):1427–1456, 1997.
Roland W. Freund and Manish Malhotra. A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides. Linear Algebra and Its Applications, 254:119–157, 1997.
Jean-Charles Gilbert. Automatic differentiation and iterative processes. Optimization Methods and Software, 1(1):13–22, 1992.
Andreas Griewank. On automatic differentiation. In Mathematical Programming: Recent Developments and Applications, pages 83–108, Amsterdam, 1989. Kluwer Academic Publishers.
Andreas Griewank. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. SIAM, Philadelphia, 2000.
Andreas Griewank, Christian Bischof, George Corliss, Alan Carle, and Karen Williamson. Derivative convergence of iterative equation solvers. Optimization Methods and Software, 2:321–355, 1993.
Andreas Griewank and George Corliss. Automatic Differentiation of Algorithms. SIAM, Philadelphia, 1991.
Andreas Griewank, David Juedes, and Jean Utke. ADOL-C, a package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical Software, 22(2):131–167, 1996.
Paul Hovland, Boyana Norris, Lucas Roh, and Barry Smith. Developing a derivative-enhanced object-oriented toolkit for scientific computations. In Michael E. Henderson, Christopher R. Anderson, and Stephen L. Lyons, editors, Object Oriented Methods for Interoperable Scientific and Engineering Computing: Proceedings of the 1998 SIAM Workshop, pages 129–137, Philadelphia, 1999. SIAM.
Dianne P. O’Leary. The block conjugated gradient algorithm and related methods. Linear Algebra and Its Applications, 29:293–322, 1980.
Louis B. Rall. Automatic Differentiation: Techniques and Applications, volume 120 of Lecture Notes in Computer Science. Springer Verlag, Berlin, 1981.
Yousef Saad. Iterative Methods for Sparse Linear Systems. PWS Publishing Company, Boston, 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bischof, C.H., Bücker, H.M., Hovland, P.D. (2000). On Combining Computational Differentiation and Toolkits for Parallel Scientific Computing. In: Bode, A., Ludwig, T., Karl, W., Wismüller, R. (eds) Euro-Par 2000 Parallel Processing. Euro-Par 2000. Lecture Notes in Computer Science, vol 1900. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44520-X_12
Download citation
DOI: https://doi.org/10.1007/3-540-44520-X_12
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67956-1
Online ISBN: 978-3-540-44520-3
eBook Packages: Springer Book Archive