Create Excel File Using OPENROWSET and XP_CMDSHELL

Create Excel File Using OPENROWSET and XP_CMDSHELL

Create PROCEDURE MCA_SP_CREATE_EXCEL @File_Name as varchar(50) = ”
AS
BEGIN
SET NOCOUNT ON
declare @File_Name as varchar(50)
DECLARE @Cmd varchar(1000)
DECLARE @fn varchar(500)
DECLARE @provider varchar(100)
DECLARE @ExcelString varchar(100)

– New File Name to be created
IF @File_Name = ”
Select @fn = ‘C:Test.xls’
ELSE
Select @fn = ‘C:’ + @File_Name + ‘.xls’

– FileCopy command string formation
SELECT @Cmd = ‘Copy C:Template.xls ‘ + @fn

– FielCopy command execution through Shell Command
EXEC MASTER..XP_CMDSHELL @cmd, NO_OUTPUT

– Mentioning the OLEDB Rpovider and excel destination filename
set @provider = ‘Microsoft.Jet.OLEDB.4.0′
set @ExcelString = ‘Excel 8.0;Database=’ + @fn

– Executing the OPENROWSET Command for copying the select contents to Excel sheet.
exec(’insert into OPENrowset(”’ + @provider + ”’,”’ + @ExcelString + ”’,”SELECT FirstName,LastName,Phone,Address,City,State,Zip FROM [Sheet1$]”)
select au_fname as FirstName,au_lname as LastName,phone,address,city,State,Zip from authors’)

exec(’insert into OPENrowset(”’ + @provider + ”’,”’ + @ExcelString + ”’,”SELECT StoreId, OrderNo, OrderDate, Quantity FROM [Sheet2$]”)
select stor_id as StoreId,Ord_Num as OrderNo,Ord_Date as OrderDate,qty as Quantity from sales’)

SET NOCOUNT OFF
END

Using XP_CMDSHELL on SQL Server 2005

So in order to use xp_cmdshell whether you are a sysadmin or a regular user you need to first enable the use of xp_cmdshell.  Doing this is similar to what was done when enabling the dedicated administrator connection.  This can be done by using sp_configure:

sp_configure ‘xp_cmdshell’, ‘1′

Next Steps

* Check whether or not you are using xp_cmdshell for previous versions of SQL Server and enable it if it is still needed when you make the upgrade to SQL 2005
* With a new installation of SQL Server 2005 check to see if you really need this option enabled.  From a security perspective it is best to minimize any risks, so keep this option disabled makes the most sense unless there is a real need.

What is XP_CMDSHELL

XP_CMDSHELL - extended stored procedure.

With this extended stored procedure you have the ability to run any command line process within your SQL script, so you can embed this within your stored procedures, jobs or batch processing.

In SQL Server 2000 this option is enabled by default, but to limit access to using xp_cmdshell only members of the sysadmin server role have default rights.

To extend rights to other users you can use the GRANT statement to allow execute rights.

With SQL Server 2005 the use of xp_cmdshell is turned off by default. As SQL Server continues to mature Microsoft has put many safeguards in place to minimize the security risks of unwanted code being executed within SQL Server or outside of SQL Server.

Examples of XP_SMDSHELL Commands

1) XP_CMDSHELL Directory Search

Let’s start out with a simple directory search example. Say you do not have access to the physical SQL Server machine, but you want to see all the files in the “C:\temp” directory that have a “.sql” extension. You can do this by issuing the following command:

exec master.dbo.xp_cmdshell ‘dir c:\temp\*.sql’

2) Create a new directory using XP_CMDSHELL

Now that wasn’t too exciting, so let’s come up with a more useful example. Imagine you are in the process of building a new SQL Agent job using Enterprise Manager. To organize the output from all the steps in this new job, you want the output of each step to go into a common directory on the server, where each step has its own output file. The only problem is the new directory does not exist. Assume you do not have access to the server, or do not want to walk down the hall to the computer room to create this new directory. No problem you can just enter the following command in Query Analyzer from the privacy of your office to accomplish creating the new directory:

exec master.dbo.xp_cmdshell ‘mkdir “c:\temp\SQL Agent Output\new_job\”‘

3) Run DTS package via stored procedure using XP_CMDSHELL

As you can see, it is very easy to execute DOS commands using xp_cmdshell. So can xp_cmdshell also run executables? The answer is YES! Suppose you have a DTS Package that you want to run via a stored procedure. There is no “sp_executeDTS” stored procedure, but you can use the following xp_cmdshell command to run your DTS package:

exec master.dbo.xp_cmdshell ‘dtsrun -E -Sserver1 -N”Export Invoices”‘

This command executed the “dtsrun” executable with the necessary parameters to run the “Export Invoices” DTS package. Are you starting to see how the xp_cmdshell extented store procedure can provide you with some alternatives and also expands the functionality you can build into a T-SQL script.

4) Execute SQL script via SQLCMD (OSQL) using XP_CMDSHELL

For my last example, suppose you have a T-SQL script stored on a hard drive that you would like to execute via a number of different stored procedures. This T-SQL script does a number of T-SQL commands and you want those same commands to be executed in many different stored procedures. In addition, the T-SQL script is generated by some other organization on a daily basis, so you are unable to include the lines of code in each stored procedure, without changing the stored procedures daily. Therefore, you want to use xp_cmdshell to execute the T-SQL script. To perform this execution you perform the following command in each of your stored procedures:

exec master.dbo.xp_cmdshell ‘osql -E -Sserver1 -i c:\temp\nightly.sql’

Here I have used xp_cmdshell to execute the SQL Server osql utility to process the T-SQL commands contained in the script file “c:\temp\nightly.sql”. See how easy it is to execute a T-SQL batch script via a stored procedure using the xp_cmdshell extended stored procedure!

5) Execute SQL script via XP_CMDSHELL only if the file exists

If you use xp_cmdshell in a script, you need to be aware that if the operating system command fails, your script will not detect that the error occurred. If you want to perform something differently if the operating system command fails then you will need to code your xp_cmdshell execution so you can capture the error. Let me demonstrate what I am talking about.

Say you want to copy an operating system file before you perform some work, but if the operating system file you want to copy does not exist to copy you do not want to perform the additional work. If you execute xp_cmdshell like so:

exec master.dbo.xp_cmdshell ‘copy c:\temp\doesnotexist.txt c:\temp\workfile.txt’Print ‘Copy worked now we can do some more stuff.’…

The “Print” statement above will be executed regardless of whether the file to be copied exists or not. Therefore, if you need to control your logic based on whether or not the “copy” command actually worked, then you should code your xp_command shell execution like this:

DECLARE @rc intEXEC @rc = master.dbo.xp_cmdshell ‘copy c:\temp\doesnotexist.txt c:\temp\workfile.txt’print @rcIF @rc <> 0BEGIN  PRINT ‘Copy Failure Skip work’ENDELSEBEGIN  Print ‘Copy worked now we can do some more stuff’  ….END

Here I captured the return code of the execution of xp_cmdshell in a variable @rc. This allows me to then check the value held by variable @rc and then I programmatically determine whether or not some additional code should be run or not.

When you use the extended stored procedure xp_cmdshell it runs commands in the background. Because of this, xp_cmdshell “MUST NOT” be used to run programs that require user input. If you try to execute a program that requires user input, the xp_cmdshell process will hang. The process hangs because the program is waiting for user input, but xp_cmdshell will never display the user interface to allow the user to enter data.

If you write lots of T-SQL scripts to automate your Database Administration task then you probably already know about the extended stored procedure xp_cmdshell. This extended stored procedure allows you additional flexibility to get at operation system resources via T-SQL, which is extremely valuable for managing your SQL Server environment. If you have not yet experimented with the power of xp_cmdshell, then take a few minutes to determine how it allows you to enter command level statements via T-SQL.

XP_CMDSHELL - Extended Stored Procedure

“xp_cmdshell” is an extended stored procedure provided by Microsoft and stored in the master database. This procedure allows you to issue operating system commands directly to the Windows command shell via T-SQL code. If needed the output of these commands will be returned to the calling routine.

Now not just anyone can run this extended stored procedure. If you want to execute this extended stored procedure, you will either need to be a member of the sysadmin role, or have the xp_sqlagent_proxy_account set up on your SQL Server. If a login executing this extended stored procedure is a member of the sysadmin role then the submitted command will run under the security context associated with the SQL Server Service account in which it runs. If the login executing this procedure is not a member of the sysadmin role, then the command uses the xp_sqlagent_proxy_account login security context for determining whether operating system commands can and cannot be run. If there is no xp_sqlagent_proxy_account then using this procedure will fail for all users not in the sysadmin role.

On the surface being able to run this extended stored procedure does not seem like much, but if the MSSQLSERVER service account has local administration rights then you can use this extended stored procedure to perform any windows operating system command. Therefore, under this circumstance the xp_cmdshell can create quite a security hole. Saying that, you need to be careful how you set up your environment to make sure someone does not corrupt your system by using the almighty powerful xp_cmdshell extended stored procedure. So let’s look at different examples of ways to use xp_cmdshell.

SQLCMD Startup Script

SQLCMD can run a startup script defined by the environment variable SQLCMDINI.

Suppose you want to always execute the following query when you are connected to the server:

SELECT @@SERVERNAME, @@VERSION

Place this command inside the init.sql file and define the following environment variable:

C:\SET SQLCMDINI=c:\sqlscripts\init.sql

Then whenever you start a SQLCMD session, the script automatically executes and displays query output listing the server name and full version string.

SQLCMD Scripting Variables

It’s a good idea to discuss scripting variables at this point before discussing different SQLCMD commands, primarily because certain commands reveal their true power only via leveraging scripting variables.

SQLCMD variables are different from T-SQL variables in many aspects. The easiest way of thinking about SQLCMD variables is that they are similar to macros that you can define in many text editors and development environments. Wherever such a variable is referenced, the query editor substitutes it with the text associated with the variable, which makes using them much more convenient then using just T-SQL variables.

Another major benefit of such variables is that they are carried forward between multiple batches. If you define a SQLCMD variable on the first batch, you can use it in all subsequent batches (unless it was undefined). Moreover, as you’ll see in the next sections, these commands can be used to parameterize queries stored in different files.

Scripting variables have the following format: $(<variable name>). The variable name is not case sensitive. Scripting variables can be defined in one of the following ways:

*      via command line option v;
*      through the :SETVAR command described later in this chapter;
*      by defining an environment variable prior to running SQLCMD.

If an environment variable name conflicts with a -v or :setvar definition or command line option, the command line option takes precedence. If a variable is referenced in a script whose value is not defined, an error message is returned and the SQLCMD session terminates. Let’s consider a very simple but illustrative example of scripting variable usage.

Suppose you want to create a generic script that backs up a database to a specific location on the disk. The following line is going to be placed in the backup.sql script file:

BACKUP DATABASE $(db) TO DISK = “$(path)\$(db).bak”

Now to use it effectively you can pass in values of two variables on the SQLCMD command line as follows:

C:>SQLCMD ic:\backup.sql -vdb=”pubs” path=”c:\data”

You can see that to back up a different database or back up to a different location, you can execute this script again and again by modifying scripting variables and not touching the script itself.

:XML [ON/OFF] SQLCMD Command

:XML [ON/OFF]

When you expect XML data to be coming back from the server, for example because of a FOR XML clause in your query, you should set XML mode to ON via this command, to properly format the output. By default it is set to OFF and thus displays unformatted data in a continuous stream.

One side effect of setting the XML mode to ON is that error messages also appear in XML format. This option does not affect formatting of any other non-XML columns present in the returned data set.

Tips and Tricks

Setting XML mode ON and OFF needs to be done only upon the start of a new batch; otherwise, results are indeterminate. Consequently, XML and regular data results cannot be mixed in the same batch or data of one of these two types becomes unreadable.

:ON ERROR [exit/ignore] SQLCMD Command

:ON ERROR [exit/ignore]

This command instructs SQLCMD on how to handle errors during script execution. The following options are available:

*      exit Exits the program with the appropriate error value
*      ignore Ignores the error and continues execution. An error message is printed.

:PERFTRACE /STDERR/STDOUT SQLCMD Command

:PERFTRACE <filename>/STDERR/STDOUT

The :PERFTRACE command redirects all performance information (see the discussion of the -p option) that by default goes to standard output.