Structure, functions, dynamics, and interactions are the basic properties to systematically understand physical systems existing in nature. In particular, there have been many scientific adventures to understand light-matter interactions, yet in the classical regime at the microscale due to the diffraction-limited optical resolution. Recently, plasmonic nano-cavity enables to induce light-matter interactions and tip-enhanced nano-spectroscopy enables to probe them at the nanoscale. However, these two approaches have developed independently with their own weaknesses so far. In this talk, I provide a novel concept of “tip-enhanced cavity-spectroscopy (TECS)” overcoming the limitations of previous approaches to induce, probe, and dynamically control ultrastrong light-matter interactions in the quantum tunneling regime. Furthermore, I provide several new directions of nano-spectroscopy and -imaging, which have not been thought in the near-field optics community before. First, we exploit extremely high tip-pressure (~GPa scale) to directly modify the lattice structure and electronic properties of materials. Second, we dynamically control the near-field polarization by adopting adaptive optics technique to near-field optics. Third, we develop conductive TECS to modify electrical properties of materials by directly flowing an electric current through the cavity junction.