I recently came to know that Oracle offers VMs with decent compute performance as part of always free cloud services, Which is great for someone like me who has too many side projects and spending way more on their hosting infrastructure than one should.

There are several great low-cost IaaS out there but their low-tier doesn't compare to Oracle Virtual Machine's specs, configuration and in some cases Oracle's free tier might be even better than their mid-tier offering.

I've been using the Oracle cloud free tier VM for two weeks, So far it's been great but there were some initial quirks while setting them up which I'm writing here to save time for others.

Quirks

Region selection

Oracle cloud services availability seems to be almost same across all regions. The region should be selected while account creation and cannot be changed later so verify that the service you need is available in the region you're choosing, Subscribing to other regions seems to be a paid feature or at least is dependent upon which home region you've chosen while creating the free tier account.

Even after successful credit card verification I wasn't able complete account creation for Hyderabad(APAC) region and after several attempts (nondescript error) I created my account with Mumbai region.

After account creation, provision of services took some time during which not all services were available and so it's advised to wait till the account provisioning is complete(Email is received when done, took about 10 minutes).

Accounts

There are several types of accounts in Oracle Cloud Infrastructure(OCI), There are two types of Oracle Cloud Accounts - Free Tier Accounts, Oracle Cloud Paid Accounts and a general Oracle Account.

For using Oracle cloud services we need a cloud account and for everything else such as Support, Documents etc. we need a general Oracle account. Knowing this can save us a lot of time and headache.

Now that we have 2 (or) 3 different oracle accounts, Managing its user credentials is tricky and care should be taken that we don't wrongly replace passwords during password update when using browser password manager.

Enabling MFA on Cloud Infrastructure Login is straight forward but doing it for Oracle Identity Cloud Service requires black magic and MFA for the general Oracle Account seems to be unavailable.

Virtual Machine

Free tier offers following options,

Two VMs with 2.0 GHz AMD EPYC™ 7551 (Naples) 2 core CPU, 1GB Memory, 0.48 Gbps Network.

Four VMs with 2.8 GHz Ampere Altra 80C ARM 1x4 core CPU, 6GBx4 Memory, 1GBps Network(Can be provisioned as 1VM or up to 4VMs).

2 Block Volumes Storage, 200 GB total (Minimum 50GB should be selected while using custom storage size in VM provisioning, Else 46GB would be provisioned by default).

AMD VM for free tier was available under 'Speciality and previous generation' and not under AMD during provisioning.

Networking seems to be limited to 50 Mbps bandwidth for the AMD VM via the Internet and 480 Mbps within the same Oracle Cloud region.

ARM VM's network bandwidth scales proportionately according to their core count.

Note: Since the ARM based instance is flexible it doesn't come under always-free category and are limited to 3,000 OCPU hours and 18,000 GB hours per month which is enough for us to run 4-Core CPU, 24 GB memory instance free(31days x 24hours x 4CPU= 2976 OCPU hours and 31days x 24hours x 24GB = 17856 GB hours memory).

Operating System

Major Operating Systems were available for provisioning, I chose Ubuntu 20.04 with the AMD VM.

I was surprised to see 50% of memory already being used by Gnome, GUI packages, snap, LXD, Oracle Cloud Agent etc. I later realized that the image is a typical Ubuntu desktop image and not the server image. I have never seen Ubuntu desktop image being used before by a cloud infrastructure provider, The server image isn't available.

I manually removed all the bloat and set the VM to multi-user.target to reduce the default memory usage to ~200MB/1GB.

On hindsight there is an option to unselect the Oracle Cloud Agent during provision which might have not installed it but other GUI bloat would have been still there. Considering the fiasco with management agents and cloud service providers it might be best to unselect it before provisioning.

Ports 80, 443 are blocked by default in the Ubuntu image. So if you are wondering why your web server is not receiving any connection even after you have opened the ports in security list of your instance like other cloud providers, that's the reason. I opened the ports using UFW, UFW doesn't work on Ubuntu image due to a Oracle's use of iptables and so I used IP tables to open port 80, 443.

  $ sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 80 -j ACCEPT
  $ sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 443 -j ACCEPT
  $ sudo iptables -L -nv
  $ sudo iptables-save

Note: Updates to the operating system changes these policy and the ports needs to be opened again.

Paid tier includes Bring Your Own Image(BYOI).

Security

Oracle claims encryption during transit(option available during VM provisioning) and at rest. The keys for encryption are managed by Oracle for the free tier, Using our own key which is stored in special hardware requires their paid vault service; Something which is available by default to everyone with Amazon AWS AWS offers HW security module for all but custom keys is a paid feature.

Although other inexpensive cloud service providers only offer full disk encryption if at all available.

That concludes the quirks I've noticed with Oracle cloud free tier VM provisioning. I received SGD 400 worth credits for 30 day trial which I haven't used as I tried only always-free services.

Support

Support Center can be accessed within the Oracle cloud dashboard(using Oracle cloud account) through which support request can be raised but to actually do anything beyond that (e.g. Attaching files) the dashboard at Oracle Cloud Support needs to be used.

I found this the hard way as the support personnel kept insisting that I need to attach screenshot, HAR from browser but there were no option to attach files. When asked, The support personnel gave me a link to a document which didn't open with the Oracle cloud account.

After further digging I found myself that there is a separate Oracle support website which requires separate Oracle account. I created a new Oracle account using the same email id and logged in it to find the issue I created using Oracle cloud account along with necessary options to upload attachments. I feel that the support personnel is under the opinion that the customer is already using this website for support request.

The issue I'm dealing with the support is explained in the next section. Trial users seem to get low priority support, Which is reasonable to me. But I doubt whether the trial users get support from trainee personnel as it seemed they didn't know their own platform as I had to educate them on multiple occasions and in the end resolved my own issue.

User reported quirks

Several Oracle Cloud Infrastructure(OCI) users have responded to this post and have mentioned their own quirks with OCI. I'm including some of the interesting ones in this section.

2 sets of VMs in free-tier or 1?

There seems to be a discrepancy among existing Oracle Cloud free tier users on whether it includes both AMD instances and ARM instances or just one of them.

I was able to create a 2nd AMD instance, 3rd AMD instance gave the free-tier limit reached error (as expected) and while creating an ARM instance my region ran out of capacity but didn't show any free-tier limit errors suggesting it's indeed 2 AMD instances and 1-4 ARM instances with flexible configuration. I have been in touch with the support to sort this out.

Another user mentioned that it's indeed 2 sets of VMs and that the ARM instances are available when there's not much traffic.

Confusingly, A VP of Oracle Cloud Infrastructure says that it's just one set of VMs in the free-tier 🤷.

Update: The support was not helpful as the support personnel kept insisting that the instances were available even though my dashboard showed the opposite. It took several days of screenshots, HAR (Network log from my browser) and then my issue was moved to the development team.

The actual response should have been, "You need to keep trying to create your ARM instance due to large demand", Because I just did that using the CLI tool and a https://www.oracle.com/security/cloud-security/key-management/faq/cron job to create an ARM instance.

I found the excellent blog post by Alexander Hitrov on this topic where he uses OCI CLI tool and cron job to try creating an instance every minute in the hopes of actually creating one when the capacity is available.

I modified the bash script a bit based on a comment on the blog as a redundant protection against over provisioning.

The script uses jq to parse the response,

#!/bin/bash
echo $(date)
if [ -e ALREADY_FINISHED ]
then
echo Already finished creating the instance
exit
fi
    OCI_OUTPUT=$(/home/[Your username]/bin/oci compute instance launch \
    --availability-domain [Your AD] \
    --compartment-id [Your compartment id] \
    --shape VM.Standard.A1.Flex \
    --subnet-id [Your subnet id] \
    --assign-private-dns-record true \
    --assign-public-ip false \
    --availability-config file:///[Your path to availabilityConfig.json] \
    --display-name my-new-instance \
    --image-id [Your image id] \
    --instance-options file:///[Your path to instanceOptions.json] \
    --shape-config file:///[Your path to shapeConfig.json \
    --ssh-authorized-keys-file /home/[Your username]/.ssh/id_rsa.pub 2>&1 | sed '1d' | jq '.code')
    echo "OCI OUPUT IS: $OCI_OUTPUT" 
if [[ $OCI_OUTPUT == \"InternalError\" ]]
then
    echo Internal Error, So trying again after a minute
exit
else
    echo Instance created
    touch ALREADY_FINISHED
fi
exit

I ran this script as a cron job every 1 minute and was able to create an ARM instance with 4-core 24GB memory under free-tier plan using this script in under 30 minutes. I can now categorically state that Oracle Cloud Infrastructure does indeed offer two sets of Virtual Machines, Two AMD VMs and 1-4 ARM VMs.

How does the block volume storage limitation apply?

Does the 2 block volume storage totalling 200GB and 46/50 GB default/minimum custom size mean we would exhaust storage limitation with just 2 VMs? An existing OCI user says that the boot volume doesn't seem to be counted towards that limitation.

What happens when the trial-period ends?

A user says their VMs were disabled after the trial period and that even the support was unavailable. Another user said that their VMs were fine but there was an attempt to charge their card.

I assume these issues were probably related to their payment methods as there are other who are successfully running always-free instances after the trial period. Anyways I will be updating the status of my VMs and account after the trial ends.

After Trial

After the trial period ends Oracle sends an email informing that the account has been disabled and that the data can be recovered within next 29 days. Instances are moved to separate compartment, I could see them only from the dashboard on the home page and not in the instances section.

Account can be upgraded to the paid account by entering the credit card details again and once the account is upgraded the instances are restored to the original compartment.

These procedures might be for preventing unexpected charges during free trial, But someone coming from other cloud providers might expect the always-free resources to continue working uninterrupted after the trial period without any user action.

Update Log

20 Oct 2022: Added iptables-save to the firewall commands.

22 Aug 2022: Updates to the operating system changes the firewall policy.

2 Jul 2022: Added note about Oracle OCPU and GB hours for ARM instance. Corrected details about firewall.

11 Feb 2022: MFA for multiple Oracle accounts.

19 Oct 2021: Added details on after trial behavior.

1 Oct 2021: Added proper link to OCI Vault service FAQ.

29 Sep 2021: Updated support, Added script to create an ARM instance.

28 Sep 2021: Accounts, Support.

27 Sep 2021: User quirks added to a separate section.

26 Sep 2021: Confirming the number of VMs available. Clarifications on AWS HW security module, Subscribing to other regions. Network bandwidth for ARM. Default storage provisioning.

25 Sep 2021: About AWS HW security module.