Traces of antimicrobials in milk are of great concern for public health. The European Union has established maximum residue limits in milk; these, however, by themselves do not guarantee the absence of drug residues in milk and related products. Currently, very little information is available on the transfer of antibiotic residues from milk to other dairy products and their potential effect on food safety. This work evaluated the presence of antibiotic residues in pasteurized fluid milk and fresh cheeses from goat’s milk containing these veterinary drugs at legal safety levels (maximum residue limits) and assessed the safety margin of these dairy products for consumers. Eight antibiotics (amoxicillin, benzylpenicillin, cloxacillin, neomycin, erythromycin, ciprofloxacin, enrofloxacin, and oxytetracycline) were selected, and three batches of fresh cheese were made from pasteurized goat’s milk spiked with each of these drugs. Drug residues in milk and cheese samples were analyzed by liquid chromatography–tandem mass spectrometry. The safety margin of goat’s milk products was calculated taking into account different age groups (children, teenagers, and adults). Results showed that most antibiotics present in raw milk remained in pasteurized milk and were transferred to cheese to a high extent; retention was above 50% in most cases. The minimum safety margin in pasteurized milk was obtained for enrofloxacin, ciprofloxacin, and erythromycin for the children’s group. For fresh cheese, an elevated safety margin was obtained for all antibiotics and age groups considered. However, the large amounts of antibiotics retained in the cheese might contribute to the development and spread of antimicrobial resistance. Considering the differences in milk from different species and the great variety of cheeses, it would be advisable to continue the traceability study of antibiotics in order to increase the safety margin of dairy products.