We have studied the domain-wall dynamics in Ta-CoFeB-MgO ultra-thin films with perpendicular magnetic anisotropy for various Co and Fe concentrations in both the amorphous and crystalline states. We observe three motion regimes with increasing magnetic field, which are consistent with a low fields creep, transitory depinning, and high fields Walker wall motion. The depinning fields are found to be as low as 2 mT, which is significantly lower than the values typically observed in 3d ferromagnetic metal films with perpendicular magnetic anisotropy. This work highlights a path toward advanced spintronics devices based on weak random pinning in perpendicular CoFeB films.