1) Configure the datadog agent to receive the APM data.
Như mình đã nói thì datadog cho chúng ta free.
Và hiện tại mình đang dụng datadog để monitor cluster k8s(home lab), log (stdout). tiếp mình sẽ setup tracing trên datadog.
Enable datadog via helm chart. thì chúng ta sẽ cần phải config helm như sau
>>>>value of helm chart datadog: # enable to receive the data APM on agent datadog. apm: enabled: true portEnabled: true
Lúc bạn sẽ thấy datadog agent sẽ có thêm port 8126 và mình cấu hình nodeport để service ở ngoài k8s cũng có thể gửi vào datadog agent.
2) Inject the APM library of datadog into Your code.
2.1) Golang
Mình cũng lấy 1 example của datadog thôi.
https://docs.datadoghq.com/tracing/trace_collection/dd_libraries/go/?tab=containers
package main import ( httptrace "gopkg.in/DataDog/dd-trace-go.v1/contrib/net/http" "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer" "net/http" "os" ) func main() { os.Setenv("DD_AGENT_HOST", "192.168.101.41") os.Setenv("DD_TRACE_AGENT_PORT", "32001") tracer.Start() defer tracer.Stop() // Create a traced mux router mux := httptrace.NewServeMux() // Continue using the router as you normally would. mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Write([]byte("Hello World!")) }) http.ListenAndServe(":8080", mux) }
Bạn sẽ thấy là mình đã thêm env để code có thể gửi về endpoint mà mình muốn.
Và khi bạn truy cập vào web golang bạn sẽ thấy trong apm của datadog sẽ hiện thị như trên
Bạn có thể tiếp tục khám phá tracing
Mình thích echo framework:
https://medium.com/qoala-engineering/datadog-apm-tracing-for-golang-service-46a6d5dce61c
2.2) Apply APM datadog for wordpress.
Bạn cũng đang thấy là mình có 1 web wordpress nè.
Giờ mình muốn tracing monitor cho wordpress của mình bằng datadog.
Còn nếu bạn muốn APM bằng Elastic thì theo dõi bài dưới.
Vì mình chạy wordpress docker nên mình sẽ cần exec vào container.
apt install wget -y wget https://github.com/DataDog/dd-trace-php/releases/download/0.86.1/datadog-php-tracer_0.86.1_amd64.deb dpkg -i datadog-php-tracer_0.86.1_amd64.deb
Nếu bạn cài trên debian thì tham kháo bài dưới:
https://www.devopsschool.com/blog/how-to-install-configure-datadog-apm-with-php-based-application-wordpress-in-linux/
Và log nó như bên dưới.
root@7d2d8c677ede:/var/www/html# dpkg -i datadog-php-tracer_0.86.1_amd64.deb (Reading database ... 17636 files and directories currently installed.) Preparing to unpack datadog-php-tracer_0.86.1_amd64.deb ... Unpacking datadog-php-tracer (0.86.1) over (0.86.1) ... Setting up datadog-php-tracer (0.86.1) ... ### PHP version PHP 7.4.20 (cli) (built: Jun 28 2021 21:43:51) ( NTS ) Copyright (c) The PHP Group Zend Engine v3.4.0, Copyright (c) Zend Technologies with Zend OPcache v7.4.20, Copyright (c), by Zend Technologies with ddtrace v0.86.1, Copyright Datadog, by Datadog ### Installing Datadog PHP tracing extension (ddtrace) ### ### Logging /usr/local/bin/php -i to a file ### Checking for extension: json Extension 'json' was found. ### Creating ddtrace.ini ### ; ***** DO NOT EDIT THIS FILE ***** ; To overwrite the INI settings for this extension, edit ; the INI file in this directory called "ddtrace-custom.ini" [datadog] extension=/opt/datadog-php/extensions/ddtrace-20190902.so datadog.trace.request_init_hook=/opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php ### ddtrace.ini created ### ### Linking ddtrace.ini for supported SAPI's ### ### Found SAPI config directory: /usr/local/etc/php/conf.d ### Linking ddtrace.ini to /usr/local/etc/php/conf.d/98-ddtrace.ini ### Linking ddtrace-custom.ini to /usr/local/etc/php/conf.d/99-ddtrace-custom.ini ### ddtrace ddtrace ### Extension enabled successfully
Bạn sẽ thấy nó thông báo là
Bạn chúng ta sẽ cần sửa file: /usr/local/etc/php/conf.d/99-ddtrace-custom.ini
Sau một hồi lần mò thì mình đã biết cách để mò cofig.
https://qiita.com/hogehoge789/items/dd8b28bdc243f4785bb7
Bước 1: Kiểm tra ddtrace đã hoạt động hay chưa?
root@7d2d8c677ede:/var/www/html# php -m | grep ddtrace ddtrace ddtrace
Output như trên là ok.
giờ bạn list config của ddtrace
root@7d2d8c677ede:/var/www/html# php --ri=ddtrace ddtrace Datadog PHP tracer extension For help, check out the documentation at https://docs.datadoghq.com/tracing/languages/php/ (c) Datadog 2020 Datadog tracing support => disabled Version => 0.86.1 DATADOG TRACER CONFIGURATION => { "date": "2023-04-03T18:14:30Z", "os_name": "Linux 7d2d8c677ede 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64", "os_version": "5.4.0-144-generic", "version": "0.86.1", "lang": "php", "lang_version": "7.4.33", "env": "production", "enabled": true, "service": "wordpress", "enabled_cli": false, "agent_url": "http:\/\/192.168.101.41:32001", "debug": false, "analytics_enabled": true, "sample_rate": 1, "sampling_rules": [], "tags": [], "service_mapping": [], "distributed_tracing_enabled": true, "priority_sampling_enabled": true, "dd_version": null, "architecture": "x86_64", "sapi": "cli", "datadog.trace.request_init_hook": "\/opt\/datadog-php\/dd-trace-sources\/bridge\/dd_wrap_autoloader.php", "open_basedir_configured": false, "uri_fragment_regex": null, "uri_mapping_incoming": null, "uri_mapping_outgoing": null, "auto_flush_enabled": true, "generate_root_span": true, "http_client_split_by_domain": false, "measure_compile_time": true, "report_hostname_on_root_span": false, "traced_internal_functions": null, "auto_prepend_file_configured": false, "integrations_disabled": "default", "enabled_from_env": false, "opcache.file_cache": null } Directive => Local Value => Master Value ddtrace.disable => Off => Off ddtrace.cgroup_file => /proc/self/cgroup => /proc/self/cgroup datadog.trace.request_init_hook => /opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php ddtrace.request_init_hook => /opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php datadog.trace.agent_url => no value => no value datadog.agent_host => 192.168.101.41 => 192.168.101.41 datadog.dogstatsd_url => no value => no value datadog.distributed_tracing => On => On datadog.dogstatsd_port => 8125 => 8125 datadog.env => production => production datadog.autofinish_spans => Off => Off datadog.trace.url_as_resource_names_enabled => On => On datadog.integrations_disabled => default => default datadog.priority_sampling => On => On datadog.service => wordpress => wordpress datadog.service_name => wordpress => wordpress datadog.service_mapping => no value => no value datadog.tags => no value => no value datadog.trace.global_tags => no value => no value datadog.trace.agent_port => 32001 => 32001 datadog.trace.analytics_enabled => On => On datadog.trace.auto_flush_enabled => On => On datadog.trace.cli_enabled => Off => Off datadog.trace.measure_compile_time => On => On datadog.trace.debug => Off => Off datadog.trace.enabled => Off => On datadog.trace.health_metrics_enabled => Off => Off datadog.trace.health_metrics_heartbeat_sample_rate => 0.001 => 0.001 datadog.trace.db_client_split_by_instance => Off => Off datadog.trace.http_client_split_by_domain => Off => Off datadog.trace.redis_client_split_by_host => Off => Off datadog.trace.memory_limit => no value => no value datadog.trace.report_hostname => Off => Off datadog.trace.resource_uri_fragment_regex => no value => no value datadog.trace.resource_uri_mapping_incoming => no value => no value datadog.trace.resource_uri_mapping_outgoing => no value => no value datadog.trace.resource_uri_query_param_allowed => no value => no value datadog.trace.http_url_query_param_allowed => * => * datadog.trace.http_post_data_param_allowed => no value => no value datadog.trace.rate_limit => 0 => 0 datadog.trace.sample_rate => 1 => 1 datadog.sampling_rate => 1 => 1 datadog.trace.sampling_rules => [] => [] datadog.span_sampling_rules => [] => [] datadog.span_sampling_rules_file => no value => no value datadog.trace.header_tags => no value => no value datadog.trace.x_datadog_tags_max_length => 512 => 512 datadog.trace.propagate_service => Off => Off datadog.trace.propagation_style_extract => tracecontext,Datadog,B3,B3 single header => tracecontext,Datadog,B3,B3 single header datadog.propagation_style_extract => tracecontext,Datadog,B3,B3 single header => tracecontext,Datadog,B3,B3 single header datadog.trace.propagation_style_inject => tracecontext,Datadog => tracecontext,Datadog datadog.propagation_style_inject => tracecontext,Datadog => tracecontext,Datadog datadog.trace.propagation_style => tracecontext,Datadog => tracecontext,Datadog datadog.trace.traced_internal_functions => no value => no value datadog.trace.agent_timeout => 500 => 500 datadog.trace.agent_connect_timeout => 100 => 100 datadog.trace.debug_prng_seed => -1 => -1 datadog.log_backtrace => Off => Off datadog.trace.generate_root_span => On => On datadog.trace.spans_limit => 1000 => 1000 datadog.trace.128_bit_traceid_generation_enabled => Off => Off datadog.trace.agent_max_consecutive_failures => 3 => 3 datadog.trace.agent_attempt_retry_time_msec => 5000 => 5000 datadog.trace.bgs_connect_timeout => 2000 => 2000 datadog.trace.bgs_timeout => 5000 => 5000 datadog.trace.agent_flush_interval => 5000 => 5000 datadog.trace.agent_flush_after_n_requests => 10 => 10 datadog.trace.shutdown_timeout => 5000 => 5000 datadog.trace.startup_logs => On => On datadog.trace.agent_debug_verbose_curl => Off => Off datadog.trace.debug_curl_output => Off => Off datadog.trace.beta_high_memory_pressure_percent => 80 => 80 datadog.trace.warn_legacy_dd_trace => On => On datadog.trace.retain_thread_capabilities => Off => Off datadog.version => no value => no value datadog.trace.obfuscation_query_string_regexp => (?i)(?:p(?:ass)?w(?:or)?d|pass(?:_?phrase)?|secret|(?:api_?|private_?|public_?|access_?|secret_?)key(?:_?id)?|token|consumer_?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)*(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)*(?::|%3A)(?:\s|%20)*(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|bearer(?:\s|%20)+[a-z0-9\._\-]|token(?::|%3A)[a-z0-9]{13}|gh[opsu]_[0-9a-zA-Z]{36}|ey[I-L](?:[\w=-]|%3D)+\.ey[I-L](?:[\w=-]|%3D)+(?:\.(?:[\w.+\/=-]|%3D|%2F|%2B)+)?|[\-]{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY[\-]{5}[^\-]+[\-]{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY|ssh-rsa(?:\s|%20)*(?:[a-z0-9\/\.+]|%2F|%5C|%2B){100,} => (?i)(?:p(?:ass)?w(?:or)?d|pass(?:_?phrase)?|secret|(?:api_?|private_?|public_?|access_?|secret_?)key(?:_?id)?|token|consumer_?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)*(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)*(?::|%3A)(?:\s|%20)*(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|bearer(?:\s|%20)+[a-z0-9\._\-]|token(?::|%3A)[a-z0-9]{13}|gh[opsu]_[0-9a-zA-Z]{36}|ey[I-L](?:[\w=-]|%3D)+\.ey[I-L](?:[\w=-]|%3D)+(?:\.(?:[\w.+\/=-]|%3D|%2F|%2B)+)?|[\-]{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY[\-]{5}[^\-]+[\-]{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY|ssh-rsa(?:\s|%20)*(?:[a-z0-9\/\.+]|%2F|%5C|%2B){100,} datadog.trace.client_ip_enabled => Off => Off datadog.trace.client_ip_header => no value => no value datadog.trace.forked_process => On => On datadog.trace.hook_limit => 100 => 100 datadog.trace.agent_max_payload_size => 52428800 => 52428800 datadog.trace.agent_stack_initial_size => 131072 => 131072 datadog.trace.agent_stack_backlog => 12 => 12 datadog.trace.propagate_user_id_default => Off => Off datadog.dbm_propagation_mode => disabled => disabled datadog.trace.cakephp_enabled => On => On datadog.trace.cakephp_analytics_enabled => Off => Off datadog.cakephp_analytics_enabled => Off => Off datadog.trace.cakephp_analytics_sample_rate => 1 => 1 datadog.cakephp_analytics_sample_rate => 1 => 1 datadog.trace.codeigniter_enabled => On => On datadog.trace.codeigniter_analytics_enabled => Off => Off datadog.codeigniter_analytics_enabled => Off => Off datadog.trace.codeigniter_analytics_sample_rate => 1 => 1 datadog.codeigniter_analytics_sample_rate => 1 => 1 datadog.trace.curl_enabled => On => On datadog.trace.curl_analytics_enabled => Off => Off datadog.curl_analytics_enabled => Off => Off datadog.trace.curl_analytics_sample_rate => 1 => 1 datadog.curl_analytics_sample_rate => 1 => 1 datadog.trace.elasticsearch_enabled => On => On datadog.trace.elasticsearch_analytics_enabled => Off => Off datadog.elasticsearch_analytics_enabled => Off => Off datadog.trace.elasticsearch_analytics_sample_rate => 1 => 1 datadog.elasticsearch_analytics_sample_rate => 1 => 1 datadog.trace.eloquent_enabled => On => On datadog.trace.eloquent_analytics_enabled => Off => Off datadog.eloquent_analytics_enabled => Off => Off datadog.trace.eloquent_analytics_sample_rate => 1 => 1 datadog.eloquent_analytics_sample_rate => 1 => 1 datadog.trace.guzzle_enabled => On => On datadog.trace.guzzle_analytics_enabled => Off => Off datadog.guzzle_analytics_enabled => Off => Off datadog.trace.guzzle_analytics_sample_rate => 1 => 1 datadog.guzzle_analytics_sample_rate => 1 => 1 datadog.trace.laravel_enabled => On => On datadog.trace.laravel_analytics_enabled => Off => Off datadog.laravel_analytics_enabled => Off => Off datadog.trace.laravel_analytics_sample_rate => 1 => 1 datadog.laravel_analytics_sample_rate => 1 => 1 datadog.trace.lumen_enabled => On => On datadog.trace.lumen_analytics_enabled => Off => Off datadog.lumen_analytics_enabled => Off => Off datadog.trace.lumen_analytics_sample_rate => 1 => 1 datadog.lumen_analytics_sample_rate => 1 => 1 datadog.trace.memcache_enabled => On => On datadog.trace.memcache_analytics_enabled => Off => Off datadog.memcache_analytics_enabled => Off => Off datadog.trace.memcache_analytics_sample_rate => 1 => 1 datadog.memcache_analytics_sample_rate => 1 => 1 datadog.trace.memcached_enabled => On => On datadog.trace.memcached_analytics_enabled => Off => Off datadog.memcached_analytics_enabled => Off => Off datadog.trace.memcached_analytics_sample_rate => 1 => 1 datadog.memcached_analytics_sample_rate => 1 => 1 datadog.trace.mongo_enabled => On => On datadog.trace.mongo_analytics_enabled => Off => Off datadog.mongo_analytics_enabled => Off => Off datadog.trace.mongo_analytics_sample_rate => 1 => 1 datadog.mongo_analytics_sample_rate => 1 => 1 datadog.trace.mongodb_enabled => On => On datadog.trace.mongodb_analytics_enabled => Off => Off datadog.mongodb_analytics_enabled => Off => Off datadog.trace.mongodb_analytics_sample_rate => 1 => 1 datadog.mongodb_analytics_sample_rate => 1 => 1 datadog.trace.mysqli_enabled => On => On datadog.trace.mysqli_analytics_enabled => Off => Off datadog.mysqli_analytics_enabled => Off => Off datadog.trace.mysqli_analytics_sample_rate => 1 => 1 datadog.mysqli_analytics_sample_rate => 1 => 1 datadog.trace.nette_enabled => On => On datadog.trace.nette_analytics_enabled => Off => Off datadog.nette_analytics_enabled => Off => Off datadog.trace.nette_analytics_sample_rate => 1 => 1 datadog.nette_analytics_sample_rate => 1 => 1 datadog.trace.pcntl_enabled => On => On datadog.trace.pcntl_analytics_enabled => Off => Off datadog.pcntl_analytics_enabled => Off => Off datadog.trace.pcntl_analytics_sample_rate => 1 => 1 datadog.pcntl_analytics_sample_rate => 1 => 1 datadog.trace.pdo_enabled => On => On datadog.trace.pdo_analytics_enabled => Off => Off datadog.pdo_analytics_enabled => Off => Off datadog.trace.pdo_analytics_sample_rate => 1 => 1 datadog.pdo_analytics_sample_rate => 1 => 1 datadog.trace.phpredis_enabled => On => On datadog.trace.phpredis_analytics_enabled => Off => Off datadog.phpredis_analytics_enabled => Off => Off datadog.trace.phpredis_analytics_sample_rate => 1 => 1 datadog.phpredis_analytics_sample_rate => 1 => 1 datadog.trace.predis_enabled => On => On datadog.trace.predis_analytics_enabled => Off => Off datadog.predis_analytics_enabled => Off => Off datadog.trace.predis_analytics_sample_rate => 1 => 1 datadog.predis_analytics_sample_rate => 1 => 1 datadog.trace.psr18_enabled => On => On datadog.trace.psr18_analytics_enabled => Off => Off datadog.psr18_analytics_enabled => Off => Off datadog.trace.psr18_analytics_sample_rate => 1 => 1 datadog.psr18_analytics_sample_rate => 1 => 1 datadog.trace.roadrunner_enabled => On => On datadog.trace.roadrunner_analytics_enabled => Off => Off datadog.roadrunner_analytics_enabled => Off => Off datadog.trace.roadrunner_analytics_sample_rate => 1 => 1 datadog.roadrunner_analytics_sample_rate => 1 => 1 datadog.trace.slim_enabled => On => On datadog.trace.slim_analytics_enabled => Off => Off datadog.slim_analytics_enabled => Off => Off datadog.trace.slim_analytics_sample_rate => 1 => 1 datadog.slim_analytics_sample_rate => 1 => 1 datadog.trace.symfony_enabled => On => On datadog.trace.symfony_analytics_enabled => Off => Off datadog.symfony_analytics_enabled => Off => Off datadog.trace.symfony_analytics_sample_rate => 1 => 1 datadog.symfony_analytics_sample_rate => 1 => 1 datadog.trace.web_enabled => On => On datadog.trace.web_analytics_enabled => Off => Off datadog.web_analytics_enabled => Off => Off datadog.trace.web_analytics_sample_rate => 1 => 1 datadog.web_analytics_sample_rate => 1 => 1 datadog.trace.wordpress_enabled => On => On datadog.trace.wordpress_analytics_enabled => Off => Off datadog.wordpress_analytics_enabled => Off => Off datadog.trace.wordpress_analytics_sample_rate => 1 => 1 datadog.wordpress_analytics_sample_rate => 1 => 1 datadog.trace.yii_enabled => On => On datadog.trace.yii_analytics_enabled => Off => Off datadog.yii_analytics_enabled => Off => Off datadog.trace.yii_analytics_sample_rate => 1 => 1 datadog.yii_analytics_sample_rate => 1 => 1 datadog.trace.zendframework_enabled => On => On datadog.trace.zendframework_analytics_enabled => Off => Off datadog.zendframework_analytics_enabled => Off => Off datadog.trace.zendframework_analytics_sample_rate => 1 => 1 datadog.zendframework_analytics_sample_rate => 1 => 1
Giờ file config sẽ như sau:
root@7d2d8c677ede:/var/www/html# cat /usr/local/etc/php/conf.d/99-ddtrace-custom.ini [ddtrace] datadog.service=wordpress datadog.service_name=nimtechnology datadog.env=production datadog.agent_host=192.168.101.41 datadog.trace.agent_port=32001 datadog.trace.analytics_enabled=true datadog.trace.auto_flush_enabled=true
Datadog tự lần mò và detect ra các service trên.