We report the discovery with the Neutron Star Interior Composition Explorer (NICER) of mHz X-ray brightness oscillations from the "clocked burster" GS 1826-238. NICER observed the source in the periods 2017 June 20-29, July 11-13, and September 9-15, for a total useful exposure of 34 ks. Two consecutive dwells obtained on 2017 September 9 revealed highly significant oscillations at a frequency of 8 mHz. The fractional, sinusoidal modulation amplitude increases from 0.7% at 1 keV to approximate to 2% at 6 keV. Similar oscillations were also detected at lower significance in three additional dwells. The oscillation frequency and amplitude are consistent with those of mHz QPOs reported in other accreting neutron star systems. A thermonuclear X-ray burst was also observed on 2017 June 22. The burst properties and X-ray colors are both consistent with GS 1826 being in a soft spectral state during these observations, findings that are confirmed by ongoing monitoring with MAXI and SWIFT-BAT. Assuming that the mHz oscillations are associated with blackbody emission from the neutron star surface, modeling of the phase-resolved spectra shows that the oscillation is consistent with being produced by modulation of the temperature component of this emission. In this interpretation, the blackbody normalization, proportional to the emitting surface area, is consistent with being constant through the oscillation cycle. We place the observations in the context of the current theory of marginally stable burning and briefly discuss the potential for constraining neutron star properties using mHz oscillations.