The Higgs boson, the mass mediator in the Standard Model (SM) of particle physics, was discovered at the Large Hadron Collider (LHC) in 2012. Since then, measuring the decays of the Higgs boson to fermions and validating the SM prediction have been one of the main physics goals of the LHC. The Higgs boson to bottom quark decay (H->bb) has been observed by the ATLAS collaboration during the LHC second data run (Run 2). Similarly, upper limits have been set on the probability of the Higgs boson decaying to charm quarks (H->cc) using ATLAS Run 2 data.

This thesis presents the latest ATLAS combined measurement of the Higgs boson decays to bottom and charm quarks using Higgs produced in association with vector bosons (VH(bb,cc)). Novel machine learning techniques are used to improve signal purity. The most stringent upper limits are set on the signal strength of VH(cc) and the charm kappa modifier (kc) to date. The observed (expected) VH(cc) signal strength upper limit is set to 11.3 (10.4) times the SM at the 95% confidence level (CL). The observed (expected) kc upper limit at the 95% CL is kc < 4.2 (4.1). The expected VH(cc) signal strength upper limit is 3 times smaller compared to the previous analysis. In addition, the measurement sets a stringent constraint on the ratio between the charm and the bottom Yukawa coupling modifiers (kc/kb) as well. The observed (expected) kc/kb upper limit at the 95% CL is |kc/kb| < 3.6 (3.5).

Improving flavor tagging performance can further improve the VH(cc) sensitivity. Including electrons and muons from b-jet and c-jet decays can improve the current flavor tagging performance by around 20%. Furthermore, the current H->cc sensitivity is limited by the available data statistics. To improve the sensitivity in the future, increasing the dataset is the key. At the HL-LHC, the ATLAS detector will collect 3000 fb^(-1) of data over 10 years, almost 20 times more data than the latest Run 2 analysis. However, to cope with the increased data rate and radiation damage, various radiation hardness tests are needed for the innermost layer of the HL-LHC ATLAS detector to inform the detector design and operation. This thesis also presents the first X-ray irradiation campaign at the Berkeley Lab on testing the prototype inner tracker readout chip, ITkPix. The X-ray dose rates are carefully calibrated and the measured ITkPix radiation damage agrees well with the expectation, paving way for future X-ray radiation testing on site for inner tracker production and operation.