Temperature, water, solar radiation, and atmospheric CO2 concentration are the main abiotic factors that are changing in the course of global warming. These abiotic factors govern the synthesis and degradation of primary (sugars, amino acids, organic acids, etc.) and secondary (phenolic and volatile flavor compounds and their precursors) metabolites directly, via the regulation of their biosynthetic pathways, or indirectly, via their effects on vine physiology and phenology. Several hundred secondary metabolites have been identified in the grape berry. Their biosynthesis and degradation have been characterized and have been shown to occur during different developmental stages of the berry. The understanding of how the different abiotic factors modulate secondary metabolism and thus berry quality is of crucial importance for breeders and growers to develop plant material and viticultural practices to maintain high-quality fruit and wine production in the context of global warming. Here, we review the main secondary metabolites of the grape berry, their biosynthesis, and how their accumulation and degradation is influenced by abiotic factors. The first part of the review provides an update on structure, biosynthesis, and degradation of phenolic compounds (flavonoids and non-flavonoids) and major aroma compounds (terpenes, thiols, methoxypyrazines, and C13 norisoprenoids). The second part gives an update on the influence of abiotic factors, such as water availability, temperature, radiation, and CO2 concentration, on berry secondary metabolism. At the end of the paper, we raise some critical questions regarding intracluster berry heterogeneity and dilution effects and how the sampling strategy can impact the outcome of studies on the grapevine berry response to abiotic factors.