Stan Math Library Release Notes
======================================================================

v2.19.1 (18 Apr 2019)
======================================================================

This is a release out of cycle to address a performance bug in v.2.19 (stan-dev/stan#2748)

New Features
------------
- GLM Optimizations, especially for small number of attributes. ( #1185 )
- Simplifies several prim/mat/fun functions using Eigen's vectorisation. (#1092)
    The following functions have been changed to use Eigen under the hood, usage remain the same:
    * add
    * add_diag
    * apply_scalar_unary
    * cholesky_corr_constrain
    * columns_dot_product
    * cumulative_sum
    * diag_post_multiply
    * diag_pre_multiply
    * divide_columns
    * elt_divide
    * elt_multiply
    * log_softmax
    * log_sum_exp
    * max
    * mean
    * min
    * prod
    * quad_form
    * quad_form_diag
    * quad_form_sym
    * rows_dot_product
    * softmax
    * subtract
    * variance
- Adds two new opencl kernels for operations matrix*vector and vector*matrix. (#1192)
- OpenCL rectangular matrix multiply optimization (#1196)

Bug Fixes
---------
- Fixing Missing bounds checks in some lpdfs (#1101)

Other
-----

- Add option of having device functions shared across opencl kernels. (#1201)
- New version of `to_size_t` to get rid of an `assert` (#1172)


v2.19.0 (20 Mar 2019)
======================================================================
Slow to arrive, but fast to compute: Stan has GPU support!

Stan 2.19 brings GPU-optimized computation to Stan users. The first supported 
function is Cholesky decomposition, the main bottleneck of many common
statistical models. Activating GPU support is easy - only a few lines are added
to the configuration and no changes have to be made to the Stan model. Cholesky
decompositions of larger matrices (including their gradients, when dealing with
parameters) are then automatically transferred to the GPU with speedups ranging
from 10 to 30, depending on matrix size and GPU.

Other GPU-optimized matrix algebra primitives and common statistical models are
soon to follow: matrix multiplication, lower triangular inverse,
eigendecomposition, GP covariance functions and several GLMs. The implementation
is based on OpenCL, so it can be used with any GPU and GPU programming-savvy
users can also add their own custom OpenCL kernels.

New Features
------------
- GPU
  - matrix multiplication (#974)
  - inverse of lower triangular matrix (#1028)
  - Operator overloading for GPU functions (#1056)
  - Cholesky decomposition (#1058)
  - specialized reverse-mode implementation for cholesky decompse (#1117)
- Host doxygen API doc on https://mc-stan.org/math/ (#500)
- Makefile completely rewritten (#581, #954, #1041, #1043, #1087)
- Adding `beta_proportion` distribution (#1018)
- adjoint vector-Jacobian product form of precomputed gradients for reverse (#876)
- Add alternative inv_logit parameterization to prevent underflow (#874)


Bug Fixes
---------
- Improved derivative for Gamma CDF w.r.t. alpha (#525)
- `value_of` incorrectly returned the wrong type (#968)
- `sum` incorrectly returned the wrong type (#987)
- `matrix_exp` incorrectly passed the argument by values (#769)
- Unit testing with Windows on Jenkins (#1046)
- `gp_cov_exp_quad` was computing the ARD mixing up rows and cols (#984)
- Fixing GoodGammaP for gcc 7.3 (#1063)

Other
-----

- Clarity on what's being tested in Math (one compiler per OS) (#943)
- Updated GitHub templates (#911)
- Improve ODE speed (#1049)
- Fix tests for threading (#1058)
- Upgrade Google Test to v1.8.1 (#1051)
- Upgrade Sundials to v4.1.0 (#1097)
- Matrix exponential action:
  - A fast implementation was implemented (#771), but it had errors (#)
  - Currently, a slow implementation is in the codebase.
- Improve the codebase:
  - Code  spacing (#587)
  - Using varidic template parameters for `return_type`, `partials_return_type`,
    and `include_summands` (#977)
  - Fixing math constants definitions for Windows (#986)
  - Avoid ambiguous instantiation of `math::sqrt()` by implementing for `double`
    and `int` (#712)
  - Clean up GPU code:
    - Seperate OpenCL kernel access into it's own class (#973)
    - `read_only` and `write_only` decorators in GPU kernels fail in Windows (#1034)
  - Fixing uninitialized values in tests:
    - `bernoulli_logit_glm_lpdf` test (#995)
    - `check_greater` test (#819)
    - `gp_exponential_cov_test` failing (#1150)
  - Updating template parameters of matern32 (#981)
  - Update `gp_dot_prod_cov` (#979)
  - Deprecating old GP covariance function names (#756)
  - Fixed compiler warnings in `test-headers` (#1110)
  - Adding required headers (#1106)
  - Turn test-math-dependencies warnings into failures on Jenkins (#1078)
  - Replace `boost::type_traits` with `std::` versions (#1126)
  - Fix doxygen errors (#1139)za
  - Clean up anaonymous namespace usage (#1006)
  - Setting STAN_NUM_THREADS to illegal value should produce an error (#947)


v.2.18.1 (24 December 2018)
======================================================================
Bugfix release - major bug in threading where certain thread count and
job size combinations could result in incorrect output. See
https://discourse.mc-stan.org/t/bug-in-map-rect-with-threading-in-stan-2-18-0/7056
for more details.

Bug Fixes
--------------------------------------------------
* Fix threading batch size bug (#1075)

v.2.18.0 (13 July 2018)
======================================================================
This is our first release with parallelism (through C++11 threads or MPI)!

New Features
--------------------------------------------------
* Support for MPI parallelization with map_rect
* Support for threaded AD and a threaded map_rect implementation (#809)
* New periodic covariance function (#921)
* Integrated Boost's 1 dimensional integrator (#913)
* Added thin QR decomposition (#900)
* New `add_diag` function for adding a real or vector to the diagonal of a matrix (#871)
* New `log_inv_logit_diff` function for the log of the difference of two inverse logits (#856)
* New dot product kernel for GPs (#834)
* Vectorized many more RNGs (#833, #722, #622)
* Add `matrix_exp_action` to calculate `exp(At)*B` (#830)
* `log_mix` vectorized (#751, #664)
* New integrators from CVODES (`integrate_ode_adams`, `_bdf`) (#735)
* New `std_normal` (#609)
* New `std_normal_log` (#728)
* New GLM primitive `normal_id_glm` (#665)
* New `chol2inv` that computes a matrix's inverse from its Cholesky factor (#649)
* New `poisson_log_glm` poisson regression with log link (#647)
* New vectorized `ordered_probit` (#645)
* New `log_modified_bessel_first_kind` that expands where Bessel 1 function can be used (#640)
* New bernoulli logit GLM (#608)

Bug Fixes
--------------------------------------------------
* Rising and falling factorial were not accepting negative arguments (#636)
* Fix normalizing constant for LKJ distribution (#628)

Other
--------------------------------------------------
* Tweaks to `effective_sample_size` for performance and upstream compatibility (#865)
* Switch to sundials package combining CVODES and IDAS, updating versions (#779, #744)
* Now publishing the Math doxygen (http://mc-stan.org/math/)
* Faster `dirichlet_lpdf` accepting more vector types (#788)
* Upgrade to Boost 1.66 (#766)
* Improved derivatives for Gamma CDF (#780)
* Sped up `multi_normal_cholesky` (#753)
* More efficient `mdivide_right_tri` (#715)
* More efficient `integrate_ode_rk45_grad_test` (#714)
* Make algebraic solver easier to use (#703, #697)
* Begin thinking about internal compatibility for complex numbers (#643)
* Faster `pow` (#642)


v.2.17.0 (05 September 2017)
======================================================================

New Features
--------------------------------------------------
* New algebraic solver! (stan-dev/stan#2023, #516)
* append_array now supports vectors of vectors

Other
--------------------------------------------------
* C++11 (and some of 14) now supported; see https://github.com/stan-dev/stan/wiki/Supported-C---Compilers-and-Language-Features
* Updated to Boost 1.64.0 (#599)
* Makefile refactoring (#602, others)
* New forward-mode test kit (#0557, #0568)
* replace copy args with refs (#346)

v.2.16.0 (15 June 2017)
======================================================================

New Features
--------------------------------------------------
* New append_array function
* Add categorical_logit_rng function

Bug Fixes
--------------------------------------------------
* Align gamma_* function parameter names with documentation

Other
--------------------------------------------------
* Update to Eigen 3.3.3
* Support g++ 4.9
* Fix overload logic in mdivide_left_tri_low so that it calls the var version of mdivide_left_tri where appropriate.
* Check consistent size of state and dy_dt in ode_system
* OperandsAndPartials refactor with new multivariate / nested container support
* Update LLT to inplace decomposition per eigen 3.3 doc
* Disable printf functions from CVODES


v.2.15.0 (13 April 2017)
======================================================================

New Features
--------------------------------------------------

* Efficient blocking algorithm for gradient of the Cholesky(#384)
* New distribution functions _lpdf / _lpmf / _lcdf / _lccdf to replace _log function (#320)
* Univariate normal distribution on sufficient statistics(#38)
* New to_matrix function for real arrays(real[]) (#467)
* New specialization of stan::math::array_builder for matrix types(#496)

Bug Fixes
--------------------------------------------------
* Fixes to hypergeometric functions(#487)

Other
--------------------------------------------------
* Speedup for categorical_rng(#503)
* Speedup for non-stiff ODE integration(#512)
* Refactor VectorView into scalar_seq_view(#464)


v.2.14.0 (26 December 2016)
======================================================================

Bug Fixes
--------------------------------------------------
* neg_binomial_2_log problem with large values of phi (#463)

Other
--------------------------------------------------
* Remove check_equal() function (#398)

v2.13.0 (25 November 2016)
======================================================================

New Features
--------------------------------------------------
* vectorization of unary functions (#347)
* specialized gradients for cove_exp_quad for amplitude and/or length-scale
  parameters and input data (#353)
* math library is consistent with Eigen 3.2.9 and 3.3.0 (#432, #429)
* added choose(n, k) function (#400)
* added bernoulli_logit_rng function (#258)

Bug Fixes
--------------------------------------------------
* beta_rng no longer returns NaN
* fixed memory issue with cholesky_decompose in reverse-mode (#415)
* no more mulitple definitions when including Stan headers in
  multiple source files (#394)
* fixed broken _MSC_VER conditionals and remove math.h includes (#370)

Other
--------------------------------------------------
* chi-square test in RNG testing pulled into utility function (#412)
* error check functions are now all void functions (#382)
* remove math.h includes (#277)
* update to Boost 1.62.0 (#435)



v2.12.0 (6 September 2016)
======================================================================

New Features
--------------------------------------------------
* update to Eigen 3.2.9 (#362)
* lots of unary functions vectorized (#202)
* reverse-mode gradients for matrix/matrix multiply,
  matrix/vector multiply, and vector/vector multiply
  received rewrites and significant speed ups for
  large matrices/vectors (#323)

Bug Fixes
--------------------------------------------------
* fixed exception handling with CVODES for g++6 (#348)
* squashed warnings with g++6 (#345)
* enforce symmetry on cov_matrix parameter inverse transform (#342)
* added int overload for log(), exp(), log2(), and log1p() to resolve
  ambiguity with g++6 (#327)
* sort_asc and sort_desc check for NaN (#253)
* wiener_log: raise exception for tau >= y (#150)
* stopped printing matrices in check functions (#146)
* added check for positive definiteness in multi_normal_rng (#44)


Other
--------------------------------------------------
* added multi-core to travis builds (#373)
* fixed makefile to recognize N_TESTS (#367)
* updated error message for check_bounded() (#332)
* made the use of stan::math namespace within the math library
  more consistent (#331)
* reimplemented integer division (#329)
* added test for multiple translation units (#328)
* updated costs for Eigen::NumTraits (#233)
* worked on getting the Math library working with Eigen 3.3-beta (#187)




v2.11.0 (27 July 2016)
======================================================================

New Features
--------------------------------------------------
* vectorization of a many unary math functions (#202)

Bug Fixes
--------------------------------------------------
* throw exception on integer divide by 0 (#321)
* allow inclusing of Stan headers in multiple translation units (#311)





v2.10.0 (15 June 2016)
======================================================================

New Features
--------------------------------------------------
* append_row and append_col for scalars (stan-dev/stan #1347)
* PYTHON2 property in makefile for 2.x requirement for
  cpplint (#300)
* refactored general ODE interface and add CVODES bdf
  integrator (#262, #293, #281, #284, #288)
* rename integrate_ode to integrate_ode_rk45 (#292)
* update to Eigen 3.2.8 (#256)
* efficiency improvement in value_of and value_of_rec (#249)
* generalize dependencies in runTests.py (#247)
* untangle include dependencies, add finer headers (#246)
* std::vector version of value_of and value_of_rec (#243)
* doc for is_vector_like and more tests (#241)
* issue and pull-request templates on GitHub (#237)
* top-level includes by shape and autodiff type (#236)
* upgrade to Boost 1.60 (#221)
* unary vectorization infrastructure (#202)
* ODE integrator fails if precision not reachable (#179)

Bug Fixes
--------------------------------------------------
* fix elementwise division argument cast problem with
  operator/(matrix, var) not compiling (#290)
* fix QR decomposition segfault for no-square matrices (#286)
* fix off-by-one bug in nested Jacobian autodiff in degenerate
  set_zero_all_adjoints_nested() call (#278)
* squash compiler warnings in OperandsAndPartials (#274)
* ifdef out residual Sundials (CVODES) print statements (#257)
* fix binomial coefficient branch to remove integer division and
  rounding (#239)
* refine clean-all make target to remove .d files (#234)
* fix Jacobian bug returning transposed result (#230)
* include stdexcept so container_view header compiles (#223)






v2.9.0 (3 December 2015)
======================================================================
New Features
--------------------------------------------------
* Added container_view (#103)
* Improved derivative calculation for beta_cdf (#124)
* Gradients computed for cholesky_decompose (#125)
* Added functions for sparse matrix multiplication (#159)
* Changed the unit_vector transformation (#204)

Bug Fixes
--------------------------------------------------
* Check inputs for cholesky_decompose (#48)
* Fixed documentation for wiener.hpp (#140)

Other
--------------------------------------------------
* Cleaned up couple_ode_system for clarity (#180)
* Additional forward mode specializations (#16, #17)
* Removed unused chaintable base class (#183)
* Updated clang++ compiler options to supress warnings (#197)
* Fixed some unit tests (#199)
* Replaced top-level using statements (#207)


v2.8.0 (8 September 2015)
======================================================================
New Features
--------------------------------------------------
* Added standard normal quantile function (#148)
* Add sparse matrix (csr format) by dense vector multiplication (#127)
* Faster derivative calculation for cholesky (#125)
* Fixes for MSVC 2008 and 2010 (#134)

Bug Fixes
--------------------------------------------------
* Fixed unit test for `mdivide_right_spd` (#153)
* Reduced compiler warnings for sing comparisons for sign functions (#151)
* Fixed gitignore for test directory (#149)
* Better assignment error messages (#146)


v2.7.0 (8 July 2015; first independent release)
======================================================================
First release
--------------------------------------------------
* Previously included as part of Stan
* Release numbering started at release number of Stan
* All work done up to and including the release of Stan Math 2.7.0
  is documented under the Stan C++ 2.7.0 release notes
