o g%j@s dZddlZddlZddlZddlZddlmZddlmZddlmZddlm Z ddlm Z dd lm Z dd lm Z dd lm Z dd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZGdddZGdddeej Z!Gdddeej"Z#GdddZ$GdddZ%Gdd d e%eZ&Gd!d"d"e%e Z'Gd#d$d$e$e Z(Gd%d&d&e$e Z)Gd'd(d(ej*Z+Gd)d*d*eZ,Gd+d,d,eZ-Gd-d.d.eZ.Gd/d0d0eZ/Gd1d2d2eZ0Gd3d4d4e Z1Gd5d6d6ee Z2e2Z3dS)7a6 .. dialect:: mssql+pyodbc :name: PyODBC :dbapi: pyodbc :connectstring: mssql+pyodbc://:@ :url: https://pypi.org/project/pyodbc/ Connecting to PyODBC -------------------- The URL here is to be translated to PyODBC connection strings, as detailed in `ConnectionStrings `_. DSN Connections ^^^^^^^^^^^^^^^ A DSN connection in ODBC means that a pre-existing ODBC datasource is configured on the client machine. The application then specifies the name of this datasource, which encompasses details such as the specific ODBC driver in use as well as the network address of the database. Assuming a datasource is configured on the client, a basic DSN-based connection looks like:: engine = create_engine("mssql+pyodbc://scott:tiger@some_dsn") Which above, will pass the following connection string to PyODBC: .. sourcecode:: text DSN=some_dsn;UID=scott;PWD=tiger If the username and password are omitted, the DSN form will also add the ``Trusted_Connection=yes`` directive to the ODBC string. Hostname Connections ^^^^^^^^^^^^^^^^^^^^ Hostname-based connections are also supported by pyodbc. These are often easier to use than a DSN and have the additional advantage that the specific database name to connect towards may be specified locally in the URL, rather than it being fixed as part of a datasource configuration. When using a hostname connection, the driver name must also be specified in the query parameters of the URL. As these names usually have spaces in them, the name must be URL encoded which means using plus signs for spaces:: engine = create_engine( "mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=ODBC+Driver+17+for+SQL+Server" ) The ``driver`` keyword is significant to the pyodbc dialect and must be specified in lowercase. Any other names passed in the query string are passed through in the pyodbc connect string, such as ``authentication``, ``TrustServerCertificate``, etc. Multiple keyword arguments must be separated by an ampersand (``&``); these will be translated to semicolons when the pyodbc connect string is generated internally:: e = create_engine( "mssql+pyodbc://scott:tiger@mssql2017:1433/test?" "driver=ODBC+Driver+18+for+SQL+Server&TrustServerCertificate=yes" "&authentication=ActiveDirectoryIntegrated" ) The equivalent URL can be constructed using :class:`_sa.engine.URL`:: from sqlalchemy.engine import URL connection_url = URL.create( "mssql+pyodbc", username="scott", password="tiger", host="mssql2017", port=1433, database="test", query={ "driver": "ODBC Driver 18 for SQL Server", "TrustServerCertificate": "yes", "authentication": "ActiveDirectoryIntegrated", }, ) Pass through exact Pyodbc string ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A PyODBC connection string can also be sent in pyodbc's format directly, as specified in `the PyODBC documentation `_, using the parameter ``odbc_connect``. A :class:`_sa.engine.URL` object can help make this easier:: from sqlalchemy.engine import URL connection_string = "DRIVER={SQL Server Native Client 10.0};SERVER=dagger;DATABASE=test;UID=user;PWD=password" connection_url = URL.create( "mssql+pyodbc", query={"odbc_connect": connection_string} ) engine = create_engine(connection_url) .. _mssql_pyodbc_access_tokens: Connecting to databases with access tokens ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Some database servers are set up to only accept access tokens for login. For example, SQL Server allows the use of Azure Active Directory tokens to connect to databases. This requires creating a credential object using the ``azure-identity`` library. More information about the authentication step can be found in `Microsoft's documentation `_. After getting an engine, the credentials need to be sent to ``pyodbc.connect`` each time a connection is requested. One way to do this is to set up an event listener on the engine that adds the credential token to the dialect's connect call. This is discussed more generally in :ref:`engines_dynamic_tokens`. For SQL Server in particular, this is passed as an ODBC connection attribute with a data structure `described by Microsoft `_. The following code snippet will create an engine that connects to an Azure SQL database using Azure credentials:: import struct from sqlalchemy import create_engine, event from sqlalchemy.engine.url import URL from azure import identity # Connection option for access tokens, as defined in msodbcsql.h SQL_COPT_SS_ACCESS_TOKEN = 1256 TOKEN_URL = "https://database.windows.net/" # The token URL for any Azure SQL database connection_string = "mssql+pyodbc://@my-server.database.windows.net/myDb?driver=ODBC+Driver+17+for+SQL+Server" engine = create_engine(connection_string) azure_credentials = identity.DefaultAzureCredential() @event.listens_for(engine, "do_connect") def provide_token(dialect, conn_rec, cargs, cparams): # remove the "Trusted_Connection" parameter that SQLAlchemy adds cargs[0] = cargs[0].replace(";Trusted_Connection=Yes", "") # create token credential raw_token = azure_credentials.get_token(TOKEN_URL).token.encode( "utf-16-le" ) token_struct = struct.pack( f"`_, stating that a connection string when using an access token must not contain ``UID``, ``PWD``, ``Authentication`` or ``Trusted_Connection`` parameters. .. _azure_synapse_ignore_no_transaction_on_rollback: Avoiding transaction-related exceptions on Azure Synapse Analytics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Azure Synapse Analytics has a significant difference in its transaction handling compared to plain SQL Server; in some cases an error within a Synapse transaction can cause it to be arbitrarily terminated on the server side, which then causes the DBAPI ``.rollback()`` method (as well as ``.commit()``) to fail. The issue prevents the usual DBAPI contract of allowing ``.rollback()`` to pass silently if no transaction is present as the driver does not expect this condition. The symptom of this failure is an exception with a message resembling 'No corresponding transaction found. (111214)' when attempting to emit a ``.rollback()`` after an operation had a failure of some kind. This specific case can be handled by passing ``ignore_no_transaction_on_rollback=True`` to the SQL Server dialect via the :func:`_sa.create_engine` function as follows:: engine = create_engine( connection_url, ignore_no_transaction_on_rollback=True ) Using the above parameter, the dialect will catch ``ProgrammingError`` exceptions raised during ``connection.rollback()`` and emit a warning if the error message contains code ``111214``, however will not raise an exception. .. versionadded:: 1.4.40 Added the ``ignore_no_transaction_on_rollback=True`` parameter. Enable autocommit for Azure SQL Data Warehouse (DW) connections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Azure SQL Data Warehouse does not support transactions, and that can cause problems with SQLAlchemy's "autobegin" (and implicit commit/rollback) behavior. We can avoid these problems by enabling autocommit at both the pyodbc and engine levels:: connection_url = sa.engine.URL.create( "mssql+pyodbc", username="scott", password="tiger", host="dw.azure.example.com", database="mydb", query={ "driver": "ODBC Driver 17 for SQL Server", "autocommit": "True", }, ) engine = create_engine(connection_url).execution_options( isolation_level="AUTOCOMMIT" ) Avoiding sending large string parameters as TEXT/NTEXT ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ By default, for historical reasons, Microsoft's ODBC drivers for SQL Server send long string parameters (greater than 4000 SBCS characters or 2000 Unicode characters) as TEXT/NTEXT values. TEXT and NTEXT have been deprecated for many years and are starting to cause compatibility issues with newer versions of SQL_Server/Azure. For example, see `this issue `_. Starting with ODBC Driver 18 for SQL Server we can override the legacy behavior and pass long strings as varchar(max)/nvarchar(max) using the ``LongAsMax=Yes`` connection string parameter:: connection_url = sa.engine.URL.create( "mssql+pyodbc", username="scott", password="tiger", host="mssqlserver.example.com", database="mydb", query={ "driver": "ODBC Driver 18 for SQL Server", "LongAsMax": "Yes", }, ) Pyodbc Pooling / connection close behavior ------------------------------------------ PyODBC uses internal `pooling `_ by default, which means connections will be longer lived than they are within SQLAlchemy itself. As SQLAlchemy has its own pooling behavior, it is often preferable to disable this behavior. This behavior can only be disabled globally at the PyODBC module level, **before** any connections are made:: import pyodbc pyodbc.pooling = False # don't use the engine before pooling is set to False engine = create_engine("mssql+pyodbc://user:pass@dsn") If this variable is left at its default value of ``True``, **the application will continue to maintain active database connections**, even when the SQLAlchemy engine itself fully discards a connection or if the engine is disposed. .. seealso:: `pooling `_ - in the PyODBC documentation. Driver / Unicode Support ------------------------- PyODBC works best with Microsoft ODBC drivers, particularly in the area of Unicode support on both Python 2 and Python 3. Using the FreeTDS ODBC drivers on Linux or OSX with PyODBC is **not** recommended; there have been historically many Unicode-related issues in this area, including before Microsoft offered ODBC drivers for Linux and OSX. Now that Microsoft offers drivers for all platforms, for PyODBC support these are recommended. FreeTDS remains relevant for non-ODBC drivers such as pymssql where it works very well. Rowcount Support ---------------- Previous limitations with the SQLAlchemy ORM's "versioned rows" feature with Pyodbc have been resolved as of SQLAlchemy 2.0.5. See the notes at :ref:`mssql_rowcount_versioning`. .. _mssql_pyodbc_fastexecutemany: Fast Executemany Mode --------------------- The PyODBC driver includes support for a "fast executemany" mode of execution which greatly reduces round trips for a DBAPI ``executemany()`` call when using Microsoft ODBC drivers, for **limited size batches that fit in memory**. The feature is enabled by setting the attribute ``.fast_executemany`` on the DBAPI cursor when an executemany call is to be used. The SQLAlchemy PyODBC SQL Server dialect supports this parameter by passing the ``fast_executemany`` parameter to :func:`_sa.create_engine` , when using the **Microsoft ODBC driver only**:: engine = create_engine( "mssql+pyodbc://scott:tiger@mssql2017:1433/test?driver=ODBC+Driver+17+for+SQL+Server", fast_executemany=True, ) .. versionchanged:: 2.0.9 - the ``fast_executemany`` parameter now has its intended effect of this PyODBC feature taking effect for all INSERT statements that are executed with multiple parameter sets, which don't include RETURNING. Previously, SQLAlchemy 2.0's :term:`insertmanyvalues` feature would cause ``fast_executemany`` to not be used in most cases even if specified. .. versionadded:: 1.3 .. seealso:: `fast executemany `_ - on github .. _mssql_pyodbc_setinputsizes: Setinputsizes Support ----------------------- As of version 2.0, the pyodbc ``cursor.setinputsizes()`` method is used for all statement executions, except for ``cursor.executemany()`` calls when fast_executemany=True where it is not supported (assuming :ref:`insertmanyvalues ` is kept enabled, "fastexecutemany" will not take place for INSERT statements in any case). The use of ``cursor.setinputsizes()`` can be disabled by passing ``use_setinputsizes=False`` to :func:`_sa.create_engine`. When ``use_setinputsizes`` is left at its default of ``True``, the specific per-type symbols passed to ``cursor.setinputsizes()`` can be programmatically customized using the :meth:`.DialectEvents.do_setinputsizes` hook. See that method for usage examples. .. versionchanged:: 2.0 The mssql+pyodbc dialect now defaults to using ``use_setinputsizes=True`` for all statement executions with the exception of cursor.executemany() calls when fast_executemany=True. The behavior can be turned off by passing ``use_setinputsizes=False`` to :func:`_sa.create_engine`. N) _MSDateTime) _MSUnicode)_MSUnicodeText)BINARY)DATETIMEOFFSET) MSDialect)MSExecutionContext) VARBINARY)JSON) JSONIndexType) JSONPathType)exc)types)util)PyODBCConnector)cursorcs0eZdZdZfddZddZddZZS)_ms_numeric_pyodbczTurns Decimals with adjusted() < 0 or > 7 into strings. The routines here are needed for older pyodbc versions as well as current mxODBC versions. cs(t||js Sfdd}|S)NcsNjrt|tjr|}|dkr|S|dkr|Sr%|S|S)Nr) asdecimal isinstancedecimalDecimaladjusted_small_dec_to_string_large_dec_to_string)valuerself super_processb/var/www/html/ecg_monitoring/venv/lib/python3.10/site-packages/sqlalchemy/dialects/mssql/pyodbc.pyprocesss  z2_ms_numeric_pyodbc.bind_processor..process)superbind_processor_need_decimal_fixrdialectr# __class__rr"r%s  z!_ms_numeric_pyodbc.bind_processorcCsBd|dkrdpddt|dddd|dDfS) Nz%s0.%s%sr-0rcSg|]}t|qSr!str).0nintr!r!r" z;_ms_numeric_pyodbc._small_dec_to_string..)absrjoinas_tuple)rrr!r!r"rs z'_ms_numeric_pyodbc._small_dec_to_stringcCs|d}dt|vr-d|dkrdpdddd|Dd |t|df}|St|d|krfd |dkr>dp?ddd d|Dd|ddd d|D|ddf}|Sd |dkrmdpndddd|Dd|df}|S)NrEz%s%s%srr+r,cSr.r!r/r1sr!r!r"r3r4z;_ms_numeric_pyodbc._large_dec_to_string..r-z%s%s.%scSr.r!r/r9r!r!r"r3r4cSr.r!r/r9r!r!r"r3r4z%s%scSr.r!r/r9r!r!r"r3r4)r7r0r6rlen)rr_intresultr!r!r"rs(  "" "z'_ms_numeric_pyodbc._large_dec_to_string)__name__ __module__ __qualname____doc__r%rr __classcell__r!r!r)r"rs  rc@ eZdZdS)_MSNumeric_pyodbcNr>r?r@r!r!r!r"rDrDc@rC)_MSFloat_pyodbcNrEr!r!r!r"rGrFrGc@seZdZdZddZdS)_ms_binary_pyodbczWraps binary values in dialect-specific Binary wrapper. If the value is null, return a pyodbc-specific BinaryNull object to prevent pyODBC [and FreeTDS] from defaulting binary NULL types to SQLWCHAR and causing implicit conversion errors. cs(jdurdSjjfdd}|S)Ncs|dur|SjjSN)dbapi BinaryNull)r DBAPIBinaryr(r!r"r#sz1_ms_binary_pyodbc.bind_processor..process)rJBinaryr'r!rLr"r%s z _ms_binary_pyodbc.bind_processorN)r>r?r@rAr%r!r!r!r"rHs rHc@seZdZdZdZddZdS)_ODBCDateTimeBindProcessorz6Add bind processors to handle datetimeoffset behaviorsFcsfdd}|S)NcsL|durdSt|tr |S|jrjsjs|S|d}tdd|}|S)Nz%Y-%m-%d %H:%M:%S.%f %zz([\+\-]\d{2})([\d\.]+)$z\1:\2)rr0tzinfotimezonehas_tzstrftimeresub)r dto_stringrr!r"r#s  z:_ODBCDateTimeBindProcessor.bind_processor..processr!r'r!rWr"r%s z)_ODBCDateTimeBindProcessor.bind_processorN)r>r?r@rArRr%r!r!r!r"rOs rOc@rC) _ODBCDateTimeNrEr!r!r!r"rXrFrXc@seZdZdZdS)_ODBCDATETIMEOFFSETTN)r>r?r@rRr!r!r!r"rYsrYc@rC)_VARBINARY_pyodbcNrEr!r!r!r"rZrFrZc@rC)_BINARY_pyodbcNrEr!r!r!r"r[ rFr[c@eZdZddZdS)_String_pyodbccC&|jdvs |jdkr|jddfS|jSN)Nmaxir)length SQL_VARCHARrrJr!r!r"get_dbapi_type z_String_pyodbc.get_dbapi_typeNr>r?r@rdr!r!r!r"r] r]c@r\)_Unicode_pyodbccCr^r_ra SQL_WVARCHARrcr!r!r"rdrez_Unicode_pyodbc.get_dbapi_typeNrfr!r!r!r"rhrgrhc@r\)_UnicodeText_pyodbccCr^r_rircr!r!r"rdrez"_UnicodeText_pyodbc.get_dbapi_typeNrfr!r!r!r"rkrgrkc@r\) _JSON_pyodbccCs |jddfS)Nrrjrcr!r!r"rd's z_JSON_pyodbc.get_dbapi_typeNrfr!r!r!r"rl&rgrlc@r\)_JSONIndexType_pyodbccC|jSrIrmrcr!r!r"rd,z$_JSONIndexType_pyodbc.get_dbapi_typeNrfr!r!r!r"rn+rgrnc@r\)_JSONPathType_pyodbccCrorIrmrcr!r!r"rd1rpz#_JSONPathType_pyodbc.get_dbapi_typeNrfr!r!r!r"rq0rgrqcs,eZdZdZfddZfddZZS)MSExecutionContext_pyodbcFcsJt|jr|jjr!t|jdr#d|_|jd7_dSdSdSdS)awhere appropriate, issue "select scope_identity()" in the same statement. Background on why "scope_identity()" is preferable to "@@identity": https://msdn.microsoft.com/en-us/library/ms190315.aspx Background on why we attempt to embed "scope_identity()" into the same statement as the INSERT: https://code.google.com/p/pyodbc/wiki/FAQs#How_do_I_retrieve_autogenerated/identity_values? rTz; select scope_identity()N) r$pre_exec_select_lastrowidr(use_scope_identityr; parameters_embedded_scope_identity statementrWr)r!r"rs8s   z"MSExecutionContext_pyodbc.pre_execcs~|jr8 z|j}Wn|jjjy|jYnw|s%|jq|d}nqt|d|_t j |_ dSt dS)NTr)rwrfetchallr(rJErrornextsetint _lastrowid_cursor_NO_CURSOR_DMLcursor_fetch_strategyr$ post_exec)rrowsrowr)r!r"rRs   z#MSExecutionContext_pyodbc.post_exec)r>r?r@rwrsrrBr!r!r)r"rr5s rrc!seZdZdZdZeZee j e j e e jeeee jeeeeee jee jee jee jee jee je e jj!e"e jj#e$e j%e j%iZ  dfdd Z&fddZ'fddZ(d d Z)dfd d Z*fddZ+Z,S)MSDialect_pyodbcTFc s^tjdd|i||jo|jot|jjd|_|jo!|dk|_||_|r-d|_ dSdS)Nuse_setinputsizesr{)rFr!) r$__init__rurJhasattrCursor_dbapi_versionr&fast_executemany!use_insertmanyvalues_wo_returning)rrrparamsr)r!r"rs  zMSDialect_pyodbc.__init__c s~z |d}Wntjyt|YSwg}td}||D]}z | t |Wq&t y:Yq&wt |S)Nz8SELECT CAST(SERVERPROPERTY('ProductVersion') AS VARCHAR)z[.\-]) exec_driver_sqlscalarr DBAPIErrorr$_get_server_version_inforTcompilesplitappendr| ValueErrortuple)r connectionrawversionrnr)r!r"rs$  z)MSDialect_pyodbc._get_server_version_infocstfdd}|S)Ncsdur||dSrI)_setup_timestampoffset_type)connrsuper_r!r" on_connectsz/MSDialect_pyodbc.on_connect..on_connect)r$r)rrr)rr"rs zMSDialect_pyodbc.on_connectcCsdd}d}|||dS)NcSs\td|}t|d|d|d|d|d|d|dd ttj|d |d d S) Nz<6hI2hrrrrirr)hoursminutes)structunpackdatetimerQ timedelta) dto_valuetupr!r!r"_handle_datetimeoffsets  zLMSDialect_pyodbc._setup_timestampoffset_type.._handle_datetimeoffsetie)add_output_converter)rrrodbc_SQL_SS_TIMESTAMPOFFSETr!r!r"rs z,MSDialect_pyodbc._setup_timestampoffset_typeNcs$|jrd|_tj||||ddS)NT)context)rr$do_executemany)rrrxrvrr)r!r"rszMSDialect_pyodbc.do_executemanycs4t||jjr|jd}|dvrdSt|||S)Nr> 010000100208001080030800708S0108S0210054HY010HYT00T)rrJrzargsr$ is_disconnect)rerrcoder)r!r"rs   zMSDialect_pyodbc.is_disconnect)FTrI)-r>r?r@supports_statement_cache supports_sane_rowcount_returningrrexecution_ctx_clsr update_copyrcolspecssqltypesNumericrDFloatrGrr[DateTimerXrrYr rZ LargeBinaryStringr]Unicoderh UnicodeTextrkr rlr rnr rqEnumrrrrrrrBr!r!r)r"ros>   r)4rArrrTrbaserrrrrrr r jsonr _MSJsonr _MSJsonIndexTyper _MSJsonPathTyper,rrrrconnectors.pyodbcrenginerr~rrrDrrGrHrOrXrYrZr[rr]rhrkrlrnrqrrrr(r!r!r!r"sTd                >: