Private user   •   9 months ago

EKS example ?

Hello
Is there any resources for using EKS and llama-3 1-nemotron-nano-8B-v1 ?
Even a docker image would be great :)

• 4 comments
• 

  Borna Rahnamay Farnod   •   9 months ago

  Hello,

  Sara shared a link to a post from Nvidia: https://github.com/NVIDIA/nim-deploy/tree/main/cloud-service-providers/aws/eks

  I also was watching a video last night. Here's its blogpost that shows how to do it on EKS.
  link to tutorial: https://awslabs.github.io/ai-on-eks/docs/blueprints/inference/GPUs/nvidia-nim-llama3

  And here's the video that walks you through the notebook, and it specifically shows you how to setup your own model: https://youtu.be/nmusho5I8jA?t=877

  just that you need to update the model information to:
  name = "llama3.1-nemotron-nano-8b-v1:latest"
  id = "nvcr.io/nim/nvidia/llama-3.1-nemotron-nano-8b-v1:latest"
  
  for the name parameter I'm not sure yet, since I haven't gone this far, you may have to try different names like "llama3-1-nemotron-nano-8b-v1:latest". let me know how it goes

• 

  Brown Carter   •   9 months ago

  Oh my Gosh, i have been trying to deploy llm eks for over 16 hours now with no progress. Geez.

  I was following this https://github.com/NVIDIA/nim-deploy/tree/main/cloud-service-providers/aws/eks but getting some errors.

• 

  Private user   •   9 months ago

  nice :)

• 

  Borna Rahnamay Farnod   •   9 months ago

  @Brown Carter, what kind of error are you running into? I got some error but it was because I was on free AWS tier...

Comments are closed.