RailwayDocs
Back to Guides

Set Up a Datadog Agent in Railway

observabilitydatadogmonitoring

Datadog provides a centralized location for logs, metrics, and traces emitted from applications deployed in various locations.

While Railway has a native, centralized logging mechanism, you may have a need to ship this data to another location, to view it alongside data collected from systems outside of Railway.

Objectives

In this tutorial you will learn how to -

  • Deploy a Datadog agent in Railway - listening for metrics, logs, and traces.
  • Configure an application to send metrics, logs, and traces to the agent.

If you are looking for a quicker way to get started, you can also deploy this project from a template.

Prerequisites

To be successful, you should already have -

Caveats

Keep in mind that the Datadog agent sends data to Datadog over the Internet, meaning you will see an increase in egress cost. If this is a concern, you may be interested in exploring self-hosted solutions, check out the OpenTelemetry Tutorial.

1. Create the project structure

First, create the project structure.

From your local machine -

  • Create a folder for your project called railway-project.
  • Create two folders inside of railway-project called agent and expressapi.

You project structure should look like this

2. Set up the Datadog agent

Now add files to the agent folder, which will build the Datadog Agent image.

  • Inside of the agent folder, create three files -
    • Dockerfile
    • syslog.yaml
    • datadog.yaml

Define the Dockerfile

Define the Dockerfile.

  • Within your Dockerfile, add the following contents.

Define the syslog.yaml file

The syslog.yaml file is used to instruct the agent to listen for syslogs to be forwarded on the configured port.

  • Within the syslog.yaml file, add the following contents -

Define the datadog.yaml file

The datadog.yaml file is used to instruct the agent to send logs to Datadog over http instead of the default tcp.

  • Within the datadog.yaml file, add the following contents -

3. Set up the Node Express app

Now build a Node Express App that will send logs and metrics to the Datadog Agent over the Private Network.

  • Create an app.js file inside of the expressapi folder you created in Step 1.

  • Use npm (or your preferred package manager) to install the required dependencies -

Define the app.js file

The app.js file defines your express server. This is where you will import the DataDog tracer and initialize the StatsD client and the Winston logger, which will send traces, metrics, and logs, respectively, to the Datadog agent.

  • Within the app.js file, add the following contents -

Winston and hot-shots

In this example app, Winston is used as the logger and hot-shots as the StatsD client.

  • Winston is configured using winston-syslog to transport logs to the Datadog agent via Syslog over udp6.
  • hot-shots is configured to send metrics to the Datadog agent over udp6.

4. Set up the Railway project

Now create the project using the CLI, then create the services and variables from within the project in Railway.

You will need your Datadog API key and Site value in this step.

If you have not already done so, please install the CLI and authenticate.

Create a project

  • In your terminal, run the following command to create a new project -

  • Name your project datadog-project when prompted (you can change this later).

  • Open your project in Railway by running the following -

Create the services

  • In Railway, create an Empty Service by clicking + New button in the top right-hand corner and choosing Empty Service in the prompt.
  • Right click on the service that is created, select Update Info and name it datadog-agent.
  • Repeat the above steps to create a second service, but name the second service expressapi.

Add the variables

Each service requires unique variables listed below. For each service, follow the steps to add the variables required for the service.

datadog-agent Variables -

expressapi Variables -

  • Click on the service card
  • Click on the Variables tab
  • Click on Raw Editor
  • Paste the required variables (be sure to update the Datadog API key and site with your own values)
  • Click Update Variables and Deploy

5. Deploy to Railway

Now you're ready to deploy the services. Use the CLI to push code from your local machine to Railway.

Railway up

Follow these steps for each service -

  • In your local terminal, change directory into the agent folder.
  • Link to datadog-project by running the following command -
  • Follow the prompts, selecting the datadog-project and production environment.
  • Link to the datadog-agent service by running the following command -
  • Follow the prompt, selecting the datadog-agent service.
  • Deploy the agent by running the following command -
  • Change directory into your expressapi folder and repeat the steps above, but for the expressapi service.

Create a domain for the Express app

The express app will send logs and metrics to the Datadog agent upon navigation to either of its two routes. So let's give it a domain -

  • Ensure that you are linked to the datadog-project and expressapi service (refer to the steps above)
  • Assign the expressapi a domain by running the following command -

6. Test and confirm

Test that your Datadog Agent is receiving and forwarding data to Datadog by navigating to the routes in the Express app -

  • /
  • /test

Generate some traffic to these two routes and verify in your Datadog instance that the data is there.

Note: it can take a few minutes to see the data in Datadog, check the Datadog Agent's logs in Railway

Bonus - add a Python service

Once you have your agent setup and working with a node app. It's easy to add more services and configure the agent to accept data from them. This bonus section quickly covers a Python implementation.

The following example uses the FastAPI Python framework.

In the main.py file, both metrics and logs are configured to be sent over StatsD and SysLog respectively -

Ensure that you configure all of the required variables in the Python service in Railway -

  • DD_AGENT_HOST - should be the private domain of the DataDog agent
  • DD_API_KEY
  • DD_AGENT_STATSD_PORT - should be 8125
  • DD_AGENT_SYSLOG_PORT - should be 515 to work with the configuration below

Update the DataDog agent's syslog.yaml file to accept data from the new source -

Conclusion

Congratulations! You have deployed a Datadog Agent and a Node Express app (and maybe a Python service) that sends logs and metrics to Datadog.

This is a very basic implementation, and you should refer to the Datadog documentation for information on how to customize the data you send.