When using a cellular phone, the accuracy of location services is dependent on how well your phone is receiving GPS signals. Depending on the type of phone you have and the construction of your home it may not work well indoors. It may work better near a window, or more certainly if you take your phone outside. It can take some time for each GPS satellite's signal to be recognized.
You may have also not given Watch Duty the ability to utilize a precise location, or location accuracy settings on your devices (android only) are not enabled, which can cause your location on the map to be off by a large margin.
Below are some resources that may help. The titles of these resources aren't fully descriptive, so please look through the links that are relevant to the type of phone you have.
About privacy and Location Services in iOS and iPadOS
Turn Location Services and GPS on or off on your iPhone or iPad
Help your phone get a more accurate location (Google Location Services a.k.a. Google Location Accuracy)