In this paper we demonstrate the ability of a derivative-driven Monte Carlo estimator to accelerate the propagation of uncertainty through two high-level non-linear finite element models. The use of derivative information amounts to a correction to the standard Monte Carlo estimation procedure that reduces the variance under certain conditions. We express the finite element models in variational form using the high-level Unified Form Language (UFL). We derive the tangent linear model automatically from this high-level description and use it to efficiently calculate the required derivative information. To study the effectiveness of the derivative-driven method we consider two stochastic PDEs; a one- dimensional Burgers equation with stochastic viscosity and a three-dimensional geometrically non-linear Mooney-Rivlin hyperelastic equation with stochastic density and volumetric material parameter. Our results show that for these problems the first-order derivative-driven Monte Carlo method is around one order of magnitude faster than the standard Monte Carlo method and at the cost of only one extra tangent linear solution per estimation problem. We find similar trends when comparing with a modern non-intrusive multi-level polynomial chaos expansion method. We parallelise the task of the repeated forward model evaluations across a cluster using the ipyparallel and mpi4py software tools. A complete working example showing the solution of the stochastic viscous Burgers equation is included as supplementary material.