Few details about his life are known, nothing at all about his formative years. Between 1693 and 1695 he served as organist of the abbey church of Saint Denis, in Paris (where his brother André de Grigny was sub-prior). It was also during that period that de Grigny studied with Nicolas Lebègue, who was by then one of the most famous French keyboard composers. In 1695 de Grigny married Marie-Magdeleine de France, daughter of a Parisian merchant. Apparently he returned to his hometown soon afterwards: the record of the birth of his first son indicates that de Grigny was already in Reims in 1696. The couple had seven children.

By late 1697 de Grigny was appointed titular organist of Notre-Dame de Reims (the exact date of the appointment is not known), the city's famous cathedral in which French kings were crowned. In 1699 the composer published his Premier livre d'orgue [contenant une messe et les hymnes des principalles festes de l'année] in Paris. De Grigny died prematurely in 1703, aged 31, shortly after accepting a job offer from Saint Symphorien, a parish church in Reims. His Livre d'orgue was reissued in 1711 through the efforts of his widow. The collection became known abroad: it was copied in 1713 by Johann Sebastian Bach, and later by Johann Gottfried Walther.