Sometimes when a container is running for a long time especially when docker logs command is called the logs dump is extra long and then a recurrent search on google for reminding how to truncate a file is mandatory for avoiding this repeating task this is the trick that it saves me from that uncomfortable long log dump.
truncate -s 0 $(docker inspect --format='{{.LogPath}}' CONTAINER_ID)Celery logs are colorized by default, the first big idea is disable color logs. It’s as easy as setting ‘CELERYD_LOG_COLOR’ to ‘False’ in ‘celery.conf’. The code could be something like this:
celery.conf.update('CELERYD_LOG_COLOR' = False)Secondly we need a function where we set up a new handler and other settings to celery logging system. For example, the code could be:
from __future__ import absolute_import from logging import BASIC_FORMAT, Formatter from logging.handlers import SysLogHandler from celery.log import redirect_stdouts_to_logger def setup_log(**args): # redirect stdout and stderr to logger redirect_stdouts_to_logger(args['logger']) # logs to local syslog hl = SysLogHandler('/dev/log') # setting log level hl.setLevel(args['loglevel']) # setting log format formatter = Formatter(BASIC_FORMAT) hl.setFormatter(formatter) # add new handler to logger args['logger'].addHandler(hl)
Pay attention to ‘redirect_stdouts_to_logger’ it’s used to send all outputs like print’s or something else to syslog.
Thirdly we want to use those settings in our celery tasks, then we have to connect ‘setup_log’ code to some celery signals. Those signals are launched when ‘task_logger’ and ‘logger’ are configured. To connect signals:
from celery.signals import after_setup_task_logger, after_setup_logger after_setup_logger.connect(setup_log) after_setup_task_logger.connect(setup_log)
Fourthly we have to get the ‘logger’, we can have more than one if we are interested in records with task context or without it. For example:
logger = get_logger('just_a_name_for_internal_use') logger_with_task_context = get_task_logger('name_of_the_task_to_be_recorded_in_logs')
Finally we only have to use those loggers with common methods DEBUG, INFO, WARN, ERROR and CRITICAL:
@celery.task def the_task(): logger.info('this is a message without task context') logger_with_task_context.debug('this record will have the prefix "name_of_the_task_to_be_recorded_in_logs" in syslog')