Skip to content
Avaloi

Nodes

See the servers that run your sites, their capacity and health, and how a server you manage joins Avaloi.

A node is a server that runs sites. Open Settings, then Nodes, to see them. You need the nodes permission to see this page.

Open it

  1. Choose Settings in the sidebar.
  2. Choose Nodes.

Each row shows the node's status, region, provider, how much of its capacity is in use, how many sites it hosts, its agent version, and its last heartbeat.

A node marked Simulated is the demo node that runs test sites at no cost.

Read a node's capacity

Select a node's name to see its capacity in CPU, memory, and disk, its addresses, and its labels. Avaloi keeps 20 percent of every node free, so the headroom you see is what sites can still use.

No servers yet

Servers start after your first paid site. Until then the list is empty. Plans cannot be bought yet, so create a test site to explore Avaloi in the meantime. See Sites list and bulk actions.

Add a server you manage

Avaloi can run sites on a server you own. The server joins through an enrollment token and a bootstrap command. Today Avaloi staff create enrollment tokens, so ask support when you want to enroll a server.

This is how a token works:

  1. Staff choose Create enrollment token, enter a node name and a region, and select Create token.
  2. Avaloi shows the token and the bootstrap command once. Copy them before the window closes, because Avaloi stores only a hash and cannot show the token again.
  3. The bootstrap command runs on your server and enrolls it.

A token works one time and expires after 15 minutes.

If your company has no plan yet, Avaloi explains that provisioning starts after you add a plan, and it points you to test sites.

When the last site on a server is removed

You do not have to do anything. When you delete the last site or environment on a shared Avaloi server, the server stops running sites at once. Avaloi waits 10 minutes, then deletes the server and its disk, so it stops costing anything.

The wait is there so that a site you delete and create again a few minutes later goes back on the same server and starts quickly. Starting a new server takes about 90 seconds. If a site arrives during the wait, the server stays.

Avaloi deletes one server at a time, writes an audit entry (node.auto_deleted) with the server's name, region, and how long it sat empty, and runs the delete as a job. Before it queues that job, Avaloi asks the server what it runs and matches the answer against its records:

  • A container that belongs to a site that exists keeps the server. Avaloi queues no delete, and the Nodes page says why (for example "Not deleted yet: 1 container still present") and when Avaloi looks again.
  • A container left behind by a delete that failed halfway, or by a site that is gone, is removed first by a reap_orphan_containers job, with a node.leftover_containers_removed audit entry. Then the server is deleted.
  • The kept copy of an environment you deleted from a site that still exists does not keep an empty server. It restores only onto an environment on the same server, so an empty server has nothing to restore into. The copy goes with the server, and its backup shows as expired. An off-site backup (restic) is the copy that outlives a server.
  • A container Avaloi cannot place at all (no site record) keeps the server until staff look at it. Avaloi never removes it by itself.

Avaloi does not retry in a loop. After a blocked or failed attempt it waits 10 minutes, then 30 minutes, 2 hours, 6 hours, and 24 hours, and it starts over when the server's containers or sites change. Node maintenance jobs never appear in a customer's job list or activity log.

Servers you manage yourself (BYOD), dedicated servers, and the simulated demo node are never deleted this way. Avaloi staff can also keep a shared server running: the Nodes page shows "Empty since" and the minutes left, and a Keep switch in the server's details.

When a server is removed outside Avaloi

An operator, or a problem at the cloud, can remove a server's virtual machine and disks without telling Avaloi. Avaloi notices on its own. Every minute it asks Google Cloud whether each shared server still exists, and it acts only on a clear answer that the machine was not found. A slow answer, a Google outage, a permission error, or a quota error never counts.

Avaloi waits for a second clear "not found" at least 5 minutes after the first. If the machine shows up again in between, Avaloi forgets the first answer. When the second answer confirms it, Avaloi:

  • Marks the server deleted, with the reason "instance removed outside Avaloi".
  • Cancels every queued, dispatched, or running job on that server. The jobs show as cancelled with the reason "This server was removed", not as failures, so nobody gets a failure notice. A delete that was waiting on the server finishes in Avaloi's records at once.
  • Marks the sites that ran on it as failed. A site owner sees that the server was removed and the site is gone, and can delete the site. The delete finishes at once, because the server holds nothing to clean up.
  • Writes an audit entry (node.removed_outside_avaloi) with the server's name and region, and tells staff.

Staff see a notice on the Nodes page. While the first answer waits for its second, the server's status shows when the cloud first could not find it. After Avaloi acts, a banner names the server and the reason for a week. Servers you manage yourself (BYOD), dedicated servers, the simulated demo node, and servers still starting are never checked. A site on a removed server cannot be recovered from that server, so restore it from an off-site backup (restic).

What a server keeps after you delete a site

Deleting a site or an environment removes its container, its data, and its web address on the server. It also removes the site's certificate files and the server's web server block for them, so nothing stays on the server.

  • Avaloi cancels any certificate or domain job that was still waiting for the site, so none runs after the delete. If a certificate order was already running, the server stops it before it writes files, because the site's container is gone.
  • The server removes the temporary address certificate, plus the certificate of any custom domain whose names all pointed at the deleted site. A certificate that also covers a name from another site stays.
  • Every 15 minutes, and when the Avaloi agent starts, the server also looks for temporary address certificates (names that start with temp-) that no site uses. The first time it finds one, it only writes a line to the agent log. It removes the files when the next check finds the same leftover, and only if the files are more than an hour old. Custom domain certificates are never removed this way.

To check a server by hand, as staff, on the server:

ls /etc/nginx/avaloi/tls/ /etc/avaloi/tls/
journalctl -u avaloi-agent --since "1 hour ago" | grep "tls sweep"

The first command should list only sites that exist. The second shows what the check found and removed.

When Google has no capacity in a location

Sometimes Google Cloud has no room for a server in a location at that moment. Avaloi does not stop at the first refusal. In one attempt, with no pause between steps, it works down this list:

  1. The preferred shape in every zone. Avaloi asks for a c4d-highmem-4 server in each zone of the location, one after another, in an order that moves one place on each attempt.
  2. The next shapes, each in every zone. If every zone is full, Avaloi tries c3d-highmem-4, then c4d-highmem-8, then c3d-highmem-8. C3D is a different Google machine family with its own pool of capacity, so it can have room when C4D has none. All four shapes use the same server image, the same Hyperdisk Balanced boot and data disks, the same network card, and the same live migration setting, and Avaloi refuses to start with a shape that does not (see the settings below). An 8 vCPU server holds about twice as many sites as a 4 vCPU server, and its disk, memory, and site count are sized from the shape it really runs on.
  3. The nearest other locations. If no shape works in any zone of the location, Avaloi moves to the nearest other location and repeats the whole list there, then to a third location if it must. That is at most three locations in all: the one you picked and two others.

A quota limit on the cloud account is different. A quota is a hard stop at once. No other shape, zone, or location can fix it, so Avaloi does not try them. Before it tries an 8 vCPU shape, Avaloi also checks how many vCPUs the cloud project has free (the project has a 12 vCPU quota in total). It skips an 8 vCPU shape that cannot fit, and it treats a project with no room for even a 4 vCPU server as a quota stop.

Which locations Avaloi may use

Data location matters to customers, so by default a location falls back only to a nearby location in the same legal area:

  • A United States location falls back to another United States location.
  • A Canada location falls back to another Canada location. It reaches the United States only if the owner adds that rule.
  • A European Union location falls back to another European Union location.
  • The United Kingdom has one location, so it falls back to the European Union.
  • Asia Pacific, the Middle East, Africa, and South America each stay in their own area. A location that is alone in its area has no fallback.

Nearest means the shortest straight line between the two cities, from a fixed table in @avaloi/fleet (NEAREST_REGIONS). Avaloi never uses a location that staff closed, one that is not in AVALOI_OPEN_REGIONS, or one where Google does not list the server shape.

Staff can change the rule with AVALOI_REGION_FALLBACK:

  • Unset or same_area is the default above.
  • off turns the fallback off, so only the location you picked is tried.
  • any lets any location back up any other, nearest first. Use it only if your customers agree that their data may cross borders.
  • A list of rules widens or narrows single areas, for example canada=canada+us,uk=uk. The areas are us, canada, mexico, south_america, uk, eu, europe_other, apac, middle_east, africa, and other. A typo stops the API at boot with a clear error.

Staff can change the shapes with GCP_NODE_SHAPE_LADDER, a comma separated list in order. The default is c4d-highmem-4,c3d-highmem-4,c4d-highmem-8,c3d-highmem-8. Every shape must be a plain C4D or C3D highmem shape with the same CPU architecture as the node image, the same disk type, the same disk interface, and the same network card. The API refuses to start with a shape that is not, and it says why.

What you are told

You are never left wondering.

  • The job's progress notes say exactly what happened. For example: "Council Bluffs is full right now. Looking for room in Columbus." and then "Council Bluffs is full right now, so we opened your data center in Columbus instead. Your plan price is unchanged." If only the server type changed, the note says "Our usual server type is full in Columbus, so we opened a different server type there. Your plan and price are unchanged."
  • Your site shows the real location. The Sites list, the site pages, the API, and the MCP tools show the city where the data center really is. Your plan keeps the price you locked, and it stays tied to the location you bought it for.
  • The owners get a bell notification and an email that names both locations, says why, and says the plan price is unchanged. The activity log of each site on the server gets a site.region_changed entry.
  • Staff see a notice on the Nodes page for a server that did not open as asked, with the first refusal, and an audit entry (node.region_fallback or node.shape_fallback).

If every shape in every zone of all three locations is full, Avaloi tries again after 2 minutes, and after 10 minutes. Then the job fails with this message: "We could not open a server for you right now. We have not charged you for anything extra. Please contact support and tell them your site name, and we will sort it out." The failed job offers a Contact support button that opens the support chat. Avaloi also opens a support ticket under your company with every location, shape, zone, and reason, and emails the owners the ticket number. The same message and ticket follow a quota stop.

If a location has no other location Avaloi may use, the message is "Google has no capacity for our servers in <city> right now. Try again in a few minutes or pick another location." There is no ticket, because you can pick another location yourself.

Your plan is safe, and nothing was created, so there is nothing to clean up. A partly created server that Google no longer has counts as removed, so staff never need to delete it by hand.

Cost and margin

Your price does not change when a server opens in another location or on another shape. Avaloi absorbs the difference. The margin stays at or above 30 percent on every shape in the list at the 70 percent target fill, for every plan, at the us-east4 list price (a unit test in region-pricing.test.ts holds that, and prints the table). An 8 vCPU server needs twice the sites to reach the same fill, so one that holds a single customer earns less than a 4 vCPU server would. A fallback location can also have a higher Google price than the one the plan was bought in. Staff see every fallback server on the Nodes page and can move sites back when capacity returns. See Region pricing.

What staff do

Starting a node through a provider, draining a node so no new sites land on it, and deleting an empty node are actions for Avaloi staff. Deleting a node through the API needs confirm: true. Staff turn the Keep switch on with PATCH /v1/nodes/{id} and {"keep": true}. If you need one of these, contact support.

Limits

  • Avaloi keeps 20 percent of every node free.
  • An enrollment token works once and expires after 15 minutes.
  • A shared server with no site is deleted after 10 minutes, unless staff keep it or a container still blocks it.
  • A server whose cloud machine is gone is marked deleted after two clear "not found" answers at least 5 minutes apart.
  • A new server opens in at most three locations (the one you picked and two nearby ones), and tries four shapes in every zone of each.
  • A fallback stays in the same legal area unless the owner widens the rule with AVALOI_REGION_FALLBACK.

Quick answers

Why is my Nodes list empty? No paid site has started a server yet. Test sites run on the simulated node.

Can I create an enrollment token myself? Not today. Avaloi staff create them. Ask support.

What does the last heartbeat mean? The last time the agent on the node reported in to Avaloi.

API

Routes you can call:

  • GET /v1/nodes
  • GET /v1/nodes/{id}
  • GET /v1/nodes/{id}/capacity

Routes for Avaloi staff:

  • POST /v1/nodes
  • POST /v1/nodes/{id}/drain
  • PATCH /v1/nodes/{id}
  • DELETE /v1/nodes/{id}
  • POST /v1/nodes/enrollment-tokens

Still stuck?

Email [email protected] with your site name and what you tried, or send us a message.