SQL Databases
Support for SQL databases like PostgreSQL, MySQL, MariaDB, SQLite or MSSQL is provided in Feathers via the @feathersjs/knex
database adapter which uses KnexJS. Knex is a fast and flexible query builder for SQL and supports many databases without the overhead of a full blown ORM like Sequelize. It still provides an intuitive syntax and more advanced tooling like migration support.
$ npm install --save @feathersjs/knex
tip
The Knex adapter implements the common database adapter API and querying syntax.
API
KnexService(options)
new KnexService(options)
returns a new service instance initialized with the given options. The following example extends the KnexService
and then uses the sqliteClient
(or relevant client for your SQL database type) from the app configuration and provides it to the Model
option, which is passed to the new MessagesService
.
import type { Params } from '@feathersjs/feathers'
import { KnexService } from '@feathersjs/knex'
import type { KnexAdapterParams, KnexAdapterOptions } from '@feathersjs/knex'
import type { Application } from '../../declarations'
import type { Messages, MessagesData, MessagesQuery } from './messages.schema'
export interface MessagesParams extends KnexAdapterParams<MessagesQuery> {}
export class MessagesService<ServiceParams extends Params = MessagesParams> extends KnexService<
Messages,
MessagesData,
ServiceParams
> {}
export const messages = (app: Application) => {
const options: KnexAdapterOptions = {
paginate: app.get('paginate'),
Model: app.get('sqliteClient'),
name: 'messages'
}
app.use('messages', new MessagesService(options))
}
Options
The Knex specific adapter options are:
Model {Knex}
(required) - The KnexJS database instancename {string}
(required) - The name of the tableschema {string}
(optional) - The name of the schema table prefix (example:schema.table
)
The common API options are:
id {string}
(optional, default:'id'
) - The name of the id field property. By design, Knex will always add anid
property.paginate {Object}
(optional) - A pagination object containing adefault
andmax
page sizemulti {string[]|boolean}
(optional, default:false
) - Allowcreate
with arrays andpatch
andremove
with idnull
to change multiple items. Can betrue
for all methods or an array of allowed methods (e.g.[ 'remove', 'create' ]
)
There are additionally several legacy options in the common API options
getModel([params])
service.getModel([params])
returns the Knex client for this table.
db(params)
service.db([params])
returns the Knex database instance for a request. This will include the schema
table prefix and use a transaction if passed in params
.
createQuery(params)
service.createQuery(params)
returns a query builder for a service request, including all conditions matching the query syntax. This method can be overriden to e.g. include associations or used in a hook customize the query and then passing it to the service call as params.knex.
app.service('messages').hooks({
before: {
find: [
async (context: HookContext) => {
const query = context.service.createQuery(context.params)
// do something with query here
query.orderBy('name', 'desc')
context.params.knex = query
}
]
}
})
params.knex
When making a service method call, params
can contain an knex
property which allows to modify the options used to run the KnexJS query. See createQuery for an example.
Querying
In addition to the common querying mechanism, this adapter also supports the following operators. Note that these operators need to be added for each query-able property to the TypeBox query schema or JSON query schema like this:
const messageQuerySchema = Type.Intersect(
[
// This will additionally allow querying for `{ name: { $ilike: 'Dav%' } }`
querySyntax(messageQueryProperties, {
name: {
$ilike: Type.String()
}
}),
// Add additional query properties here
Type.Object({})
],
{ additionalProperties: false }
)
$like
Find all records where the value matches the given string pattern. The following query retrieves all messages that start with Hello
:
app.service('messages').find({
query: {
text: {
$like: 'Hello%'
}
}
})
Through the REST API:
/messages?text[$like]=Hello%
$notlike
The opposite of $like
; resulting in an SQL condition similar to this: WHERE some_field NOT LIKE 'X'
app.service('messages').find({
query: {
text: {
$notlike: '%bar'
}
}
})
Through the REST API:
/messages?text[$notlike]=%bar
$ilike
For PostgreSQL only, the keywork $ilike can be used instead of $like to make the match case insensitive. The following query retrieves all messages that start with hello
(case insensitive):
app.service('messages').find({
query: {
text: {
$ilike: 'hello%'
}
}
})
Through the REST API:
/messages?text[$ilike]=hello%
Search
Basic search can be implemented with the query operators.
Associations
While resolvers offer a reasonably performant way to fetch associated entities, it is also possible to join tables to populate and query related data. This can be done by overriding the createQuery method and using the Knex join methods to join the tables of related services.
Querying
Considering a table like this:
await db.schema.createTable('todos', (table) => {
table.increments('id')
table.string('text')
table.bigInteger('personId').references('id').inTable('people').notNullable()
return table
})
To query based on properties from the people
table, join the tables you need in createQuery
like this:
class TodoService<ServiceParams = KnexAdapterParams<TodoQuery>> extends KnexService<Todo> {
createQuery(params: KnexAdapterParams<AdapterQuery>) {
const query = super.createQuery(params)
query.join('people as person', 'todos.personId', 'person.id')
return query
}
}
This will alias the table name from people
to person
(since our Todo only has a single person) and then allow to query all related properties as dot separated properties like person.name
, including the Feathers query syntax:
// Find the Todos for all Daves older than 100
app.service('todos').find({
query: {
'person.name': 'Dave',
'person.age': { $gt: 100 }
}
})
Note that in most applications, the query-able properties have to explicitly be added to the TypeBox query schema or JSON query schema. Support for the query syntax for a single property can be added with the queryProperty
helper:
import { queryProperty } from '@feathersjs/typebox'
export const todoQueryProperties = Type.Pick(userSchema, ['text'])
export const todoQuerySchema = Type.Intersect(
[
querySyntax(userQueryProperties),
// Add additional query properties here
Type.Object(
{
// Only query the name for strings
'person.name': Type.String(),
// Support the query syntax for the age
'person.age': queryProperty(Type.Number())
},
{ additionalProperties: false }
)
],
{ additionalProperties: false }
)
Populating
Related properties from the joined table can be added as aliased properties with query.select:
class TodoService<ServiceParams = KnexAdapterParams<TodoQuery>> extends KnexService<Todo> {
createQuery(params: KnexAdapterParams<AdapterQuery>) {
const query = super.createQuery(params)
query
.join('people as person', 'todos.personId', 'person.id')
// This will add a `personName` property
.select('person.name as personName')
// This will add a `person.age' property
.select('person.age')
return query
}
}
important
Since SQL does not have a concept of nested objects, joined properties will be dot separated strings, not nested objects. Conversion can be done by e.g. using Lodash _.set
in a resolver converter.
This works well for individual properties, however if you require the complete (and safe) representation of the entire related data, use a resolver instead.
Transactions
The Knex adapter comes with three hooks that allows to run service method calls in a transaction. They can be used as application wide hooks or per service.
To use the transactions feature, you must ensure that the three hooks (start, end and rollback) are being used.
At the start of any request, a new transaction will be started. All the changes made during the request to the services that are using knex will use the transaction. At the end of the request, if successful, the changes will be commited. If an error occurs, the changes will be forfeit, all the creates
, patches
, updates
and deletes
are not going to be commited.
The object that contains transaction
is stored in the params.transaction
of each request.
Important
If you call another Knex service within a hook and want to share the transaction you will have to pass context.params.transaction
in the parameters of the service call.
Sometimes it can be important to know when the transaction has been completed (committed or rolled back). For example, we might want to wait for transaction to complete before we send out any realtime events. This can be done by awaiting on the transaction.committed
promise which will always resolve to either true
in case the transaction has been committed, or false
in case the transaction has been rejected.
app.service('messages').publish(async (data, context) => {
const { transaction } = context.params
if (transaction) {
const success = await transaction.committed
if (!success) {
return []
}
}
return app.channel(`rooms/${data.roomId}`)
})
This also works with nested service calls and nested transactions. For example, if a service calls transaction.start()
and passes the transaction param to a nested service call, which also calls transaction.start()
in it's own hooks, they will share the top most committed
promise that will resolve once all of the transactions have successfully committed.
Example Transaction Setup
We will be using TypeBox schemas throughout, but that is not a requirement.
We will have two services Order
and ShippingOrder
When we create an Order
we want to automatically create a ShippingOrder
, but if Order
or ShippingOrder
fail to be created we want to roll everything back and not save either.
Order Schema
export const orderSchema = Type.Object(
{
id: Type.String({ format: 'uuid' }),
item: Type.String(),
address: Type.String(),
quantity: Type.Number()
},
{ $id: 'Order', additionalProperties: false }
)
Shipping Order Schema
export const shippingOrderSchema = Type.Object(
{
id: Type.String({ format: 'uuid' }),
order_id: Type.String({ format: 'uuid', $schema: 'Order' }),
expedited: Type.Boolean(),
shipped: Type.Boolean()
},
{ $id: 'ShippingOrder', additionalProperties: false }
)
After hook
Let's start by adding our logic to automatically create our ShippingOrder
.
In our order.ts
file we can add this hook
after: {
create: [
async (context: HookContext<OrderService>) => {
const ourOrder = context.result as Order //Let's not deal with arrays or pagination for now
await context.app
.service(shippingOrderPath)
.create({ expedited: true, shipped: false, order_id: ourOrder.id })
}
]
}
The problem
Now that we have our logic in, Order
will automatically create ShippingOrder
. But what if something goes wrong and the Order
is created but ShippingOrder
isn't. This could cause an order to never be shipped.
We can solve this problem in two ways outlined below.
tip
You can emulate an error by throwing an error in the before create hook of your shipping-order.ts
file
create: [
async () => {
throw new Error('Fail')
},
schemaHooks.validateData(shippingOrderDataValidator),
schemaHooks.resolveData(shippingOrderDataResolver)
]
Application wide wrapping transaction
Using the global hooks in src/app.ts
we are able to wrap all of our create
, update
, and patch
hooks.
const transactionHandler = async (context: HookContext<any>, next: NextFunction) => {
try {
console.log('Start our work')
await transaction.start()(context)
await next()
await transaction.end()(context)
console.log('Work done')
} catch (err) {
console.log('Rollback')
await transaction.rollback()(context)
throw err
}
}
// Register hooks that run on all service methods
app.hooks({
around: {
create: [transactionHandler],
patch: [transactionHandler],
update: [transactionHandler],
delete: [transactionHandler]
}
})
What this does is for any create
/update
/patch
/delete
request, we are starting a transaction that will be available in context.params.transaction
.
Note this does not mean we are done, when a create
request is made to Order
, it will have context.params.transaction
available to it but we have to pass that along to ShippingOrder
create request.
Let's revisit our hook that automatically creates ShippingOrder
and modify it to pass our transaction with the request.
after: {
create: [
async (context: HookContext<OrderService>) => {
const ourOrder = context.result as Order
await context.app.service(shippingOrderPath).create(
{ expedited: true, shipped: false, order_id: ourOrder.id },
{ transaction: context.params.transaction } // <--
)
}
]
}
tip
We have to use await here otherwise the transaction will close before the creation is finished. For something like sending an email, you can opt to not await.context.params.transaction?.committed.then((success: any) => {
if (!success) return
//Send Email
})
Service wide wrapping transaction
The simplest way of doing this is
- Add
transaction.start()
in the before create hook. - Add
transaction.end()
in the after create hook. - Add
transaction.rollback()
in the error all hook.
app.service(orderPath).hooks({
around: {
// ...
},
before: {
// ...
create: [
schemaHooks.validateData(orderDataValidator),
schemaHooks.resolveData(orderDataResolver),
transaction.start()
]
},
after: {
create: [
async (context: HookContext<OrderService>) => {
const ourOrder = context.result as Order //Let's not deal with arrays or pagination for now
await context.app
.service(shippingOrderPath)
.create(
{ expedited: true, shipped: false, order_id: ourOrder.id },
{ transaction: context.params.transaction }
)
},
transaction.end()
]
},
error: {
all: [transaction.rollback()]
}
})
Example with around hook
When utilizing the around hook, you must pass the context manually. Remember to handle your errors as well, since around
hooks will not throw into the error
hook
{
around: {
create: [
async (context: HookContext<OrderService>, next: NextFunction) => {
console.log('Start Work')
await transaction.start()(context)
try {
//We can do any work here, similar to a before hook
await next()
const ourOrder = context.result as Order
await context.app
.service(shippingOrderPath)
.create(
{ expedited: true, shipped: false, order_id: ourOrder.id },
{ transaction: context.params.transaction }
)
console.log('End Work')
transaction.end()(context)
} catch (err) {
console.log('Rollback')
transaction.rollback()(context)
throw err
}
}
]
}
}
Error handling
The adapter only throws Feathers Errors with the message to not leak sensitive information to a client. On the server, the original error can be retrieved through a secure symbol via import { ERROR } from '@feathersjs/knex'
import { ERROR } from 'feathers-knex'
try {
await knexService.doSomething()
} catch (error: any) {
// error is a FeathersError with just the message
// Safely retrieve the Knex error
const knexError = error[ERROR]
}
Migrations
In a generated application, migrations are already set up. See the CLI guide and the KnexJS migrations documentation for more information.