Enables calls to the Google Cloud's Vertex AI API to access Large Language Models in a chat-like fashion.

To use, you will need to have one of the following authentication methods in place:

  • You are logged into an account permitted to the Google Cloud project using Vertex AI.
  • You are running this on a machine using a service account permitted to the Google Cloud project using Vertex AI.
  • The GOOGLE_APPLICATION_CREDENTIALS environment variable is set to the path of a credentials file for a service account permitted to the Google Cloud project using Vertex AI.
const model = new ChatGoogleVertexAI({
temperature: 0.7,
});
const result = await model.invoke("What is the capital of France?");

Hierarchy

  • BaseChatGoogleVertexAI<GoogleAuthOptions>
    • ChatGoogleVertexAI

Constructors

Properties

connection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAIChatInstance, GoogleVertexAIChatPrediction, GoogleAuthOptions<JSONClient>>
examples: ChatExample[] = []

Help the model understand what an appropriate response is

maxOutputTokens: number = 1024

Maximum number of tokens to generate in the completion.

model: string = "chat-bison"

Model to use

streamedConnection: GoogleVertexAILLMConnection<BaseLanguageModelCallOptions, GoogleVertexAIChatInstance, GoogleVertexAIChatPrediction, GoogleAuthOptions<JSONClient>>
temperature: number = 0.2

Sampling temperature to use

topK: number = 40

Top-k changes how the model selects tokens for output.

A top-k of 1 means the selected token is the most probable among all tokens in the model’s vocabulary (also called greedy decoding), while a top-k of 3 means that the next token is selected from among the 3 most probable tokens (using temperature).

topP: number = 0.8

Top-p changes how the model selects tokens for output.

Tokens are selected from most probable to least until the sum of their probabilities equals the top-p value.

For example, if tokens A, B, and C have a probability of .3, .2, and .1 and the top-p value is .5, then the model will select either A or B as the next token (using temperature).

Methods

""