By the way, doctor: Are MRI contrast agents harmful?

Do the contrast agents used with MRI scans have side effects? I have had several MRIs and am wondering if there’s reason to be worried.