Avoid rebuilding callback field list on every serialization loop (#66343)

https://github.com/apache/airflow/commit/f3a3d4e4194797f66c07ae3d52d4fc295c82cc5d