Flower
×

Worker:

celery@worker_crawl

Worker pool options
Implementation Celery.concurrency.prefork:TaskPool
Max concurrency 1
Processes 1051945
Max tasks per child 4
Put guarded by semaphore False
Timeouts 0, 0
Writes {'total': 126938, 'avg': '1.00', 'all': '1.00', 'raw': '126938', 'strategy': 'fair', 'inqueues': {'total': 1, 'active': 0}}
Worker PID 1
Prefetch Count 4
Pool size control
Broker options
Hostname frege-redis
Userid None
Virtual host /
Port 6379
Insist False
SSL False
Transport redis
Connect timeout 4
Transport options {}
Login method None
URI prefix None
Heartbeat 120.0
Failover strategy round-robin
Alternates []

Active queues being consumed from

Name Exclusive Durable Routing key No ACK Alias Queue arguments Binding arguments Auto delete
crawl False True crawl False None None None False

Processed number of completed tasks

frege.repositories.tasks.crawl_repos_task 101551

Active currently executing tasks

Name UUID Ack PID args kwargs
frege.repositories.tasks.crawl_repos_task 44991625-3f11-493d-bd57-0367ac874769 True 1051945 ['BitbucketIndexer'] {}

Scheduled scheduled (eta/countdown/retry) tasks

Name UUID args kwargs

Reserved tasks that have been received, but are still waiting to be executed

Name UUID args kwargs
frege.repositories.tasks.crawl_repos_task 9512844f-7b24-4986-aeb6-c539a47f74c7 ['SourceforgeIndexer'] {}
frege.repositories.tasks.crawl_repos_task c048d330-05ab-4431-89f5-fab0b5974100 ['BitbucketIndexer'] {}
frege.repositories.tasks.crawl_repos_task 3af1ba20-75ad-4a10-9b32-2c6d0c3e6178 ['GitLabIndexer'] {}
frege.repositories.tasks.crawl_repos_task d59e2977-6ef4-4ce5-81e5-2141e21ec5cd ['GitHubIndexer'] {}

Revoked cancelled tasks

UUID

Task limits

Task name Rate limit Timeouts
frege.repositories.tasks.analyze_commit_message_quality_task
frege.repositories.tasks.analyze_file_task
frege.repositories.tasks.crawl_repos_task
frege.repositories.tasks.process_repo_task
health_check.contrib.celery.tasks.add
Configuration options
ABSOLUTE_URL_OVERRIDES {}
ADMINS []
ALLOWED_HOSTS ['api.localhost', '.localhost', '127.0.0.1']
APPEND_SLASH True
ASGI_APPLICATION frege.asgi.application
AUTHENTICATION_BACKENDS ['django.contrib.auth.backends.ModelBackend']
AUTH_PASSWORD_VALIDATORS ********
AUTH_USER_MODEL auth.User
BASE_DIR PosixPath('/app')
CACHES {'default': {'BACKEND': 'django.core.cache.backends.redis.RedisCache', 'LOCATION': 'redis://frege-redis:6379/'}}
CACHE_MIDDLEWARE_ALIAS default
CACHE_MIDDLEWARE_KEY_PREFIX ********
CACHE_MIDDLEWARE_SECONDS 600
CELERY_ACCEPT_CONTENT ['application/json']
CELERY_BROKER_URL redis://frege-redis:6379//
CELERY_CACHE_BACKEND redis://frege-redis:6379/
CELERY_RESULT_BACKEND redis://frege-redis:6379/
CELERY_TASK_ROUTES {'frege.repositories.tasks.process_repo_task': {'queue': 'downloads'}, 'frege.repositories.tasks.crawl_repos_task': {'queue': 'crawl'}}
CELERY_TASK_SERIALIZER json
CHANNEL_LAYERS {'default': {'BACKEND': 'channels_redis.core.RedisChannelLayer', 'CONFIG': {'hosts': [['frege-redis', '6379']]}}}
CORS_ALLOWED_ORIGINS ['http://localhost:3000', 'http://localhost:3001', 'http://localhost:3030']
CORS_ALLOW_CREDENTIALS True
CORS_ALLOW_METHODS ['GET', 'POST', 'OPTIONS']
CSRF_COOKIE_AGE 31449600
CSRF_COOKIE_HTTPONLY False
CSRF_COOKIE_NAME csrftoken
CSRF_COOKIE_PATH /
CSRF_COOKIE_SAMESITE Lax
CSRF_COOKIE_SECURE False
CSRF_FAILURE_VIEW django.views.csrf.csrf_failure
CSRF_HEADER_NAME HTTP_X_CSRFTOKEN
CSRF_TRUSTED_ORIGINS []
CSRF_USE_SESSIONS False
DATABASES {'default': {'ENGINE': 'django.db.backends.postgresql', 'NAME': 'frege', 'USER': 'frege', 'PASSWORD': '********', 'HOST': 'frege-postgres', 'PORT': '5432', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'CONN_HEALTH_CHECKS': False, 'OPTIONS': {}, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'MIGRATE': True, 'MIRROR': None, 'NAME': None}}}
DATABASE_ROUTERS ********
DATA_UPLOAD_MAX_MEMORY_SIZE 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS 1000
DATA_UPLOAD_MAX_NUMBER_FILES 100
DATETIME_FORMAT N j, Y, P
DATETIME_INPUT_FORMATS ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M']
DATE_FORMAT N j, Y
DATE_INPUT_FORMATS ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG True
DEBUG_PROPAGATE_EXCEPTIONS False
DECIMAL_SEPARATOR .
DEFAULT_AUTO_FIELD django.db.models.BigAutoField
DEFAULT_CHARSET utf-8
DEFAULT_EXCEPTION_REPORTER django.views.debug.ExceptionReporter
DEFAULT_EXCEPTION_REPORTER_FILTER django.views.debug.SafeExceptionReporterFilter
DEFAULT_FROM_EMAIL webmaster@localhost
DEFAULT_INDEX_TABLESPACE
DEFAULT_TABLESPACE
DISALLOWED_USER_AGENTS []
DJANGO_CORS_ALLOWED_HOSTS ['http://localhost:3030']
DOWNLOAD_DIR_MAX_SIZE_BYTES 2000000000
DOWNLOAD_PATH /var/tmp/frege/
DOWNLOAD_TASK_NAME celery@worker_downloads
EMAIL_BACKEND django.core.mail.backends.smtp.EmailBackend
EMAIL_HOST localhost
EMAIL_HOST_PASSWORD ********
EMAIL_HOST_USER
EMAIL_PORT 25
EMAIL_SSL_KEYFILE ********
EMAIL_SUBJECT_PREFIX [Django]
EMAIL_USE_LOCALTIME False
EMAIL_USE_SSL False
EMAIL_USE_TLS False
EXTERNAL_APPS ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django_extensions', 'rest_framework', 'rest_framework_api_key', 'corsheaders', 'django_filters', 'channels', 'health_check', 'health_check.db', 'health_check.cache', 'health_check.storage', 'health_check.contrib.migrations', 'health_check.contrib.celery', 'health_check.contrib.celery_ping', 'health_check.contrib.psutil', 'health_check.contrib.redis']
FILE_UPLOAD_HANDLERS ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE 2621440
FILE_UPLOAD_PERMISSIONS 420
FIRST_DAY_OF_WEEK 0
FIXTURE_DIRS []
FORMS_URLFIELD_ASSUME_HTTPS False
FORM_RENDERER django.forms.renderers.DjangoTemplates
HEALTHCHECK_CELERY_PING_TIMEOUT 1
HEALTHCHECK_CELERY_QUEUE_TIMEOUT 1
HEALTHCHECK_CELERY_TIMEOUT 1
HEALTH_CHECK {'DISK_USAGE_MAX': 70, 'MEMORY_MIN': 100, 'WARNINGS_AS_ERRORS': True, 'SUBSETS': {}, 'DISABLE_THREADING': False}
IGNORABLE_404_URLS []
INSTALLED_APPS ['django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django_extensions', 'rest_framework', 'rest_framework_api_key', 'corsheaders', 'django_filters', 'channels', 'health_check', 'health_check.db', 'health_check.cache', 'health_check.storage', 'health_check.contrib.migrations', 'health_check.contrib.celery', 'health_check.contrib.celery_ping', 'health_check.contrib.psutil', 'health_check.contrib.redis', 'frege', 'frege.repositories.apps.RepositoriesConfig', 'frege.indexers.apps.IndexersConfig', 'frege.analyzers.apps.AnalyzersConfig']
INTERNAL_IPS []
LANGUAGES [['af', 'Afrikaans'], ['ar', 'Arabic'], ['ar-dz', 'Algerian Arabic'], ['ast', 'Asturian'], ['az', 'Azerbaijani'], ['bg', 'Bulgarian'], ['be', 'Belarusian'], ['bn', 'Bengali'], ['br', 'Breton'], ['bs', 'Bosnian'], ['ca', 'Catalan'], ['ckb', 'Central Kurdish (Sorani)'], ['cs', 'Czech'], ['cy', 'Welsh'], ['da', 'Danish'], ['de', 'German'], ['dsb', 'Lower Sorbian'], ['el', 'Greek'], ['en', 'English'], ['en-au', 'Australian English'], ['en-gb', 'British English'], ['eo', 'Esperanto'], ['es', 'Spanish'], ['es-ar', 'Argentinian Spanish'], ['es-co', 'Colombian Spanish'], ['es-mx', 'Mexican Spanish'], ['es-ni', 'Nicaraguan Spanish'], ['es-ve', 'Venezuelan Spanish'], ['et', 'Estonian'], ['eu', 'Basque'], ['fa', 'Persian'], ['fi', 'Finnish'], ['fr', 'French'], ['fy', 'Frisian'], ['ga', 'Irish'], ['gd', 'Scottish Gaelic'], ['gl', 'Galician'], ['he', 'Hebrew'], ['hi', 'Hindi'], ['hr', 'Croatian'], ['hsb', 'Upper Sorbian'], ['hu', 'Hungarian'], ['hy', 'Armenian'], ['ia', 'Interlingua'], ['id', 'Indonesian'], ['ig', 'Igbo'], ['io', 'Ido'], ['is', 'Icelandic'], ['it', 'Italian'], ['ja', 'Japanese'], ['ka', 'Georgian'], ['kab', 'Kabyle'], ['kk', 'Kazakh'], ['km', 'Khmer'], ['kn', 'Kannada'], ['ko', 'Korean'], ['ky', 'Kyrgyz'], ['lb', 'Luxembourgish'], ['lt', 'Lithuanian'], ['lv', 'Latvian'], ['mk', 'Macedonian'], ['ml', 'Malayalam'], ['mn', 'Mongolian'], ['mr', 'Marathi'], ['ms', 'Malay'], ['my', 'Burmese'], ['nb', 'Norwegian Bokmål'], ['ne', 'Nepali'], ['nl', 'Dutch'], ['nn', 'Norwegian Nynorsk'], ['os', 'Ossetic'], ['pa', 'Punjabi'], ['pl', 'Polish'], ['pt', 'Portuguese'], ['pt-br', 'Brazilian Portuguese'], ['ro', 'Romanian'], ['ru', 'Russian'], ['sk', 'Slovak'], ['sl', 'Slovenian'], ['sq', 'Albanian'], ['sr', 'Serbian'], ['sr-latn', 'Serbian Latin'], ['sv', 'Swedish'], ['sw', 'Swahili'], ['ta', 'Tamil'], ['te', 'Telugu'], ['tg', 'Tajik'], ['th', 'Thai'], ['tk', 'Turkmen'], ['tr', 'Turkish'], ['tt', 'Tatar'], ['udm', 'Udmurt'], ['ug', 'Uyghur'], ['uk', 'Ukrainian'], ['ur', 'Urdu'], ['uz', 'Uzbek'], ['vi', 'Vietnamese'], ['zh-hans', 'Simplified Chinese'], ['zh-hant', 'Traditional Chinese']]
LANGUAGES_BIDI ['he', 'ar', 'ar-dz', 'ckb', 'fa', 'ug', 'ur']
LANGUAGE_CODE en-us
LANGUAGE_COOKIE_HTTPONLY False
LANGUAGE_COOKIE_NAME django_language
LANGUAGE_COOKIE_PATH /
LANGUAGE_COOKIE_SECURE False
LOCALE_PATHS []
LOGGING {}
LOGGING_CONFIG logging.config.dictConfig
LOGIN_REDIRECT_URL /accounts/profile/
LOGIN_URL /accounts/login/
MANAGERS []
MAX_DOWNLOAD_TASKS_COUNT 24
MEDIA_ROOT /app/media
MEDIA_URL /media/
MESSAGE_STORAGE django.contrib.messages.storage.fallback.FallbackStorage
MIDDLEWARE ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'corsheaders.middleware.CorsMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'whitenoise.middleware.WhiteNoiseMiddleware']
MIGRATION_MODULES {}
MONTH_DAY_FORMAT F j
NUMBER_GROUPING 0
PASSWORD_HASHERS ********
PASSWORD_RESET_TIMEOUT ********
PREPEND_WWW False
PROJECT_APPS ['frege', 'frege.repositories.apps.RepositoriesConfig', 'frege.indexers.apps.IndexersConfig', 'frege.analyzers.apps.AnalyzersConfig']
REDIS_HOST frege-redis
REDIS_PERSISTENCE_ENABLED False
REDIS_PORT 6379
REDIS_URL redis://frege-redis:6379/
REST_FRAMEWORK {'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination', 'PAGE_SIZE': 100, 'DEFAULT_PERMISSION_CLASSES': ['rest_framework.permissions.IsAuthenticated'], 'DEFAULT_AUTHENTICATION_CLASSES': ['rest_framework.authentication.TokenAuthentication', 'rest_framework.authentication.SessionAuthentication'], 'DEFAULT_FILTER_BACKENDS': ['django_filters.rest_framework.DjangoFilterBackend'], 'DEFAULT_THROTTLE_CLASSES': ['frege.utils.throttling.ApiKeyRateThrottle'], 'DEFAULT_THROTTLE_RATES': {'apikey': '********'}}
ROOT_URLCONF frege.urls
SECRET_KEY ********
SECRET_KEY_FALLBACKS ********
SECURE_CONTENT_TYPE_NOSNIFF True
SECURE_CROSS_ORIGIN_OPENER_POLICY same-origin
SECURE_HSTS_INCLUDE_SUBDOMAINS False
SECURE_HSTS_PRELOAD False
SECURE_HSTS_SECONDS 0
SECURE_REDIRECT_EXEMPT []
SECURE_REFERRER_POLICY same-origin
SECURE_SSL_REDIRECT False
SERVER_EMAIL root@localhost
SESSION_CACHE_ALIAS default
SESSION_COOKIE_AGE 1209600
SESSION_COOKIE_HTTPONLY True
SESSION_COOKIE_NAME sessionid
SESSION_COOKIE_PATH /
SESSION_COOKIE_SAMESITE Lax
SESSION_COOKIE_SECURE False
SESSION_ENGINE django.contrib.sessions.backends.db
SESSION_EXPIRE_AT_BROWSER_CLOSE False
SESSION_SAVE_EVERY_REQUEST False
SESSION_SERIALIZER django.contrib.sessions.serializers.JSONSerializer
SETTINGS_MODULE frege.settings
SHORT_DATETIME_FORMAT m/d/Y P
SHORT_DATE_FORMAT m/d/Y
SIGNING_BACKEND django.core.signing.TimestampSigner
SILENCED_SYSTEM_CHECKS []
STATICFILES_DIRS []
STATICFILES_FINDERS ['django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder']
STATIC_ROOT /app/static
STATIC_URL /static/
STORAGES {'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'}, 'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage'}}
TEMPLATES [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages']}}]
TEST_NON_SERIALIZED_APPS []
TEST_RUNNER django.test.runner.DiscoverRunner
THOUSAND_SEPARATOR ,
TIME_FORMAT P
TIME_INPUT_FORMATS ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE UTC
USE_I18N True
USE_THOUSAND_SEPARATOR False
USE_TZ True
USE_X_FORWARDED_HOST False
USE_X_FORWARDED_PORT False
WSGI_APPLICATION frege.wsgi.application
X_FRAME_OPTIONS DENY
YEAR_MONTH_FORMAT F Y
broker_connection_retry_on_startup True
include ['frege.repositories.tasks', 'celery.app.builtins', 'health_check.contrib.celery.tasks']
is_overridden <bound method Settings.is_overridden of <Settings "frege.settings">>
worker_max_tasks_per_child 4
System usage statistics
utime 8528.149623
stime 1132.595906
maxrss 183684
ixrss 0
idrss 0
isrss 0
minflt 645274783
majflt 369
nswap 0
inblock 69047
oublock 264448
msgsnd 0
msgrcv 0
nsignals 0
nvcsw 6355933
nivcsw 142243
Other statistics
uptime 1660071