Add second-order derivative functions

[gerlero]

Draft PR for changes discussed in JuliaDiff/ForwardDiff.jl#678

Fixes #71.

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (211b675) 82.94% compared to head (72e4201) 83.37%.
Report is 2 commits behind head on master.

Files	Patch %	Lines
src/AbstractDifferentiation.jl	93.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #122      +/-   ##
==========================================
+ Coverage   82.94%   83.37%   +0.42%     
==========================================
  Files           8        8              
  Lines         428      451      +23     
==========================================
+ Hits          355      376      +21     
- Misses         73       75       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gdalle]

I think the change is welcome here and coherent regardless of whether it makes it into ForwardDiff. However it probably needs tests too, which I guess is why the PR remains a draft.
Thank you so much for taking the time!

[gerlero]

@gdalle Thanks. Tests are now live and passing so this is ready for review

[gerlero]

Gentle bump @gdalle; can you be a reviewer in this PR?

[gdalle]

Thanks for the contribution!
The backend-agnostic part looks good to me. I would have to read up on ForwardDiff tags to understand what you did in the relevant extension, so if someone else can review that part great, if not I'll do it too.
Any reason why you didn't add tests for Tracker and RuleConfig backends?

[gerlero]

@gdalle @devmotion Thanks for the reviews.

Any reason why you didn't add tests for Tracker and RuleConfig backends?

I simply omitted them because I saw that those backends don't have tests for hessian either. I've now tried adding the tests: RuleConfig passes, but Tracker fails (ArgumentError: Tracker backend does not support nested differentiation.). So, I've now added the RuleConfig backend tests, but left Tracker out.

EDIT: the RuleConfig tests fail with Julia nightly though..

[gdalle]

As soon as I understand what's going on with the tags, it's good to merge on my end

[gerlero]

Thanks. Can we get a new release with this feature?

[devmotion]

JuliaRegistries/General#100271