compare two lists with PowerShell

I am asked occasionally to compare a list of employee ID numbers to find the differences or the matches. Notepad++ does not do a great job of this, so I put together a quick PowerShell solution.

$dupes = @()
[System.Collections.ArrayList]$arrA = Get-Content U:\listA.txt
[System.Collections.ArrayList]$arrB = Get-Content U:\listB.txt
foreach ($itemA in $arrA) {
if ($arrB -match $itemA) {
$arrB.Remove($itemA)
$dupes += $itemA
}
}

Now $arrB contains only items from listB.txt that do not also appear on listA.txt. Also $dupes contains the items that exist in both files.

Posted in Uncategorized | Leave a comment

Intel Smart Response can’t be enabled with Windows 8.1 / 2012 R2

Scenario: Single 500GB HDD and single 240GB SSD.  I wanted to use SSD cache feature to accelerate hard disk access.  Both HDD and SSD are initialized as GPT, with the “system” disk containing a Recovery Partition, an EFI System Partition, and the C: drive (Boot Partition), with the entire disk allocated.  The SSD is shown as Unallocated.  Intel’s Rapid Storage Technology would not let me enable their Smart Response Technology (SSD caching), however.  The only performance accelerator in the Rapid Storage Technology UI was for Dynamic Storage Accelerator.

On my HP EliteDesk 800 G1 SFF, I had to change the disk access mode in the UEFI/BIOS from AHCI to RAID.  Windows then failed to boot (no surprise there).  After switching back to AHCI, I followed the advice from the following post to reboot in Safe Mode to enable RAID access, and that worked.  If I recall correctly, in Windows’ Device Manager, there was no obvious Intel-provided driver under Storage Controllers until after I enabled RAID mode – only a “Microsoft Storage Spaces Controller.”  I obtained the most recent version of the driver from Intel, but was still unable to enable SSD caching.
http://www.eightforums.com/installation-setup/24141-convert-ahci-mode-raid-mode-without-re-installing.html

After following advice from Tom_GPT on the following thread, I shrunk my C: drive by 1GB (that size was arbitrary).  This resulted in 1GB of unallocated space at the end of the 500GB hard disk.  I was then able to launch Intel’s RST and enable Smart Response Technology.  Using Intel’s latest versions of both their RST and storage drivers is probably advisable.
https://communities.intel.com/thread/45540?start=15&tstart=0

Posted in Uncategorized | Leave a comment

Office 365: removing Litigation Hold mailboxes in an Exchange Hybrid environment

In our hybrid Exchange 2010 / Exchange Online environment, we occasionally need to place an Exchange 2010 mailbox on Litigation Hold. In some cases, that user’s mailbox will need to be removed but the Active Directory account will need to be retained. Exchange 2010 will not allow a mailbox on Litigation Hold to be removed, so our practice has been to simply export the mailbox to PST for retention, manually remove the Litigation Hold, and then remove the mailbox. However, we’ve learned that Exchange Online requires a slight change to that procedure.

Exchange Online was reporting an error regarding a few such users.
Exchange: An unknown error has occurred. Refer to correlation ID:

Referencing this article to help determine the problem, I ran some code against MSOL to look at a more detailed error report.
http://support2.microsoft.com/kb/2741233

$errors = (Get-MsolUser –UserPrincipalName user@domain.edu).Errors
$errors | foreach-object {"`nService: " + $_.ErrorDetail.Name.split("/")[0]; "Error Message: " + $_.ErrorDetail.ObjectErrors.ErrorRecord.ErrorDescription}

The output provided the details need to understand the problem.

Service: exchange
Error Message: Exchange can't disable the mail user "NAMPRXXXXXX.prod.outlook.com/Microsoft Exchange Hosted Organizations/tenant.onmicrosoft.com/user" because it is on litigation hold.

First I tried to simply remove the MsolUser using this command.
Remove-MailUser –Identity name@domain.com –IgnoreLegalHold

However, that returned an error.

The following error occurred during validation in agent 'Windows LiveId Agent': 'Unable to perform the save operation. 'user' is not within a valid server write scope.'

After engaging Microsoft on the problem, we determined there are two options to address the error:

  1. If the MSOL account is not actually required in Azure Active Directory (AAD), we can simply delete it and purge it from the AAD recycle bin. At the next DirSync cycle, a new MsolUser will be created and the error will be resolved. (See the important NOTE below.)
  2. An Exchange Online license could be assigned temporarily to the MsolUser to create a new Exchange Online mailbox. After allowing time for the mailbox to be created plus additional time for a DirSync cycle, remove the Exchange Online license again, and the mailbox will be deleted. This should allow the backend processing to occur and resolve the error.

In most cases, Option 1 is probably most palatable. I issued these two commands, and after the regular DirSync scheduled sync, the error has been resolved. Of course you can add the -Force parameter to quickly execute the commands without having to confirm.

NOTE: If the AAD account is removed, this will also remove the user’s access to other Office 365 data such as OneDrive for Business.

Remove-MsolUser -UserPrincipalName user@domain.edu
Remove-MsolUser -UserPrincipalName user@domain.edu -RemoveFromRecycleBin

In summary, the way to avoid the problem is to remove the Litigation Hold from the Exchange 2010 mailbox, then wait for a DirSync cycle, and then remove the Exchange 2010 mailbox. If both actions are taken quickly together and an error is reported in the Office 365 Admin Center, just purge the AAD account as described above to resolve the error.

Posted in Uncategorized | Leave a comment

NetScaler Login exceeds maximum allowed users after 10.1 upgrade

Shortly after our recent NetScaler upgrade from 9.3 -> 10.1, users reported getting the error “Login exceeds maximum allowed users” in their browsers when attempting to log in to the Access Gateway (NetScaler Gateway).  A remote session with a Citrix technician revealed that we had indeed hit our license limit as seen under NetScaler Gateway / Active User Sessions. We did see that some users were logged in two or more times, and it’s possible that the way licenses are consumed under 10.1 is different from 9.3, which might be why we never hit the licensing limit before.  The options presented by the Citrix tech were:

  1. Ask users to deliberately log out of the Access Gateway when they are done (vs. just allowing their sessions to time out) in order to free up their license.  This would, of course, require user education.
  2. Switch our Access Gateway Virtual Server from SmartAccess Mode (includes VPN access) to Basic Mode (ICA proxy-only).  Without taking additional steps such as allowing VPN for just a subset of our users, this option would remove VPN ability for all users from the gateway but allow unlimited connections through the gateway to our apps.
  3. Lower the timeout value for our Access Gateway, forcing users to re-authenticate to the gateway during the workday.

If memory serves, the technician also mentioned that the 10.5 version of NetScaler would allow a user who logged into the Access Gateway more than once to “assume” the license from his/her previous session.  An immediate upgrade to 10.5 was not an option in our case.

After a quick review of our environment, the technician suggested we switch to Basic Mode on our Virtual Server under NetScaler Gateway / Virtual Servers as no VPN was required in our environment.

Posted in Uncategorized | Leave a comment

NetScaler Integrated Caching behavior after 9.3 -> 10.1 upgrade

After a recent NetScaler upgrade from 9.3 to 10.1, we noticed a change in the behavior of the Integrated Caching feature.  Integrated Caching had been enabled for the previous two years, but with the Memory Usage Limit set to zero, caching had been effectively disabled.  After the upgrade, our PeopleSoft application began displaying incorrect content after users logged in.

We were able to tell that Integrated Caching was delivering cached content by visiting Optimization / Integrated Caching / Content Groups and seeing both “non-304 Hits” and “304 Hits” for the DEFAULT Content Group, along with a non-zero value under Memory Usage.

Integrated-Caching-10-1

Since we run in HA mode, we could consult our not-yet-upgraded, 9.3 NetScaler node.  Visiting Integrated Caching / Content Groups / DEFAULT revealed the expected values of zero for Memory Usage, Non-304 Hits, and 304 Hits.

Integrated-Caching-9-3

 

Our solution was to disable Integrated Caching in System / Settings / Configure Basic Features as it wasn’t needed.  As soon as we did this, the undesired content stopped displaying within our PeopleSoft application.

Integrated-Caching-Disable-10-1

Posted in Uncategorized | Leave a comment

Rotate images in ADFS 3.0

ADFS 3.0 is otherwise known as ADFS 2012 R2 since it is available only on Server 2012 R2. As I gain some experience with it, one of the nice configuration options is the ability to use PowerShell to customize the sign-in page.

Among the customizations we’ve made is one to help keep our sign-in page from looking stale over time. I wrote this simple PowerShell script to rotate the large “illustration” image occasionally. It runs as a Scheduled Task, and pulls approved images randomly from a file system folder. The script also logs which image was in place at any given time in case that happens to be interesting to someone at some point.

cd X:\path\images
$RandomImage = Get-ChildItem | Get-Random | %{((Get-Item $_).VersionInfo).FileName}
(Get-Date -format G) + " $RandomImage" | Out-File X:\path\Logs\IllustrationRandomizer.log -Append
Set-AdfsWebTheme -TargetName Custom_Theme -Illustration @{path=$RandomImage}

Posted in Uncategorized | Tagged | Leave a comment

Redirect URL for SSL_BRIDGE Virtual Server on NetScaler

When you create an SSL_BRIDGE Virtual Server (VIP) in NetScaler, there is no way to specify a Redirect URL (the field is grayed out).  So if your back-end servers are down, there’s no way to specify an outage page.  If you try to create a Responder policy as a workaround, you will be unable to bind it to the SSL_BRIDGE Virtual Server if it that policy contains anything other than a DROP or RESET action (http://discussions.citrix.com/topic/336769-ssl-bridge-virtual-server-response-when-down/).

You can, however, use a Listen Policy to deliver your outage page.  Create a new SSL Virtual Server alongside the existing SSL_BRIDGE Virtual Server using the same IP address and port.  (You can’t normally do this, but you can when you specify a Listen Policy on the Advanced tab.)

Example steps for setting up the new SSL Virtual Server in version 9.3 of the GUI (no changes are made to the existing SSL_BRIDGE Virtual Server):

  • Add your new SSL Virtual Server using the same IP and same port as the existing SSL_BRIDGE Virtual Server
  • Do not bind any Services to the new SSL Virtual Server (it will always be DOWN)
  • Set http://outage_page.your_domain.com (or whatever you please) as the Redirect URL on the new SSL Virtual Server
  • Bind a Listen Policy to the new SSL Virtual Server by setting a Listen Priority of 1 and a Listen Policy Rule of SYS.VSERVER(“ssl_bridge_virtual_server_name”).STATE.NE(up)
  • Add the same SSL certificate that is bound to the SSL_BRIDGE Virtual Server to the new SSL Virtual Server

Click for larger image:

Capture

References: http://support.citrix.com/article/CTX139276 and http://support.citrix.com/proddocs/topic/ns-system-10-map/cb-br-aws-acc-encryp-mapi-smb-ssl-br-win-dom-tsk.html

Posted in Uncategorized | Leave a comment

Exchange 2010 Mailbox Move fails with MapiExceptionCallFailed

Moving a mailbox within a single Exchange 2010 SP2 RU8 server failed repeatedly with error text like the following.  Dismounting/remounting the database and running various New-MailboxRepairRequest commands did not fix the issue or provide guidance.

Error: MapiExceptionCallFailed: IExchangeFastTransferEx.TransferBuffer failed (hr=0x80004005, ec=1162)
Diagnostic context:
Lid: 55847 EMSMDBPOOL.EcPoolSessionDoRpc called [length=3004]
Lid: 43559 EMSMDBPOOL.EcPoolSessionDoRpc returned [ec=0x0][length=685][latency=15]
Lid: 23226 — ROP Parse Start —
Lid: 27962 ROP: ropFXDstCopyConfig [83]
Lid: 27962 ROP: ropTellVersion [134]
Lid: 27962 ROP: ropFXDstPutBufferEx [157]
Lid: 17082 ROP Error: 0x48A
Lid: 31329
Lid: 21921 StoreEc: 0x48A
Lid: 27962 ROP: ropExtendedError [250]
Lid: 1494 —- Remote Context Beg —-
Lid: 1238 Remote Context Overflow
Lid: 1947 StoreEc: 0x48A
[…edited…]
Lid: 1750 —- Remote Context End —-
Lid: 26849
Lid: 21817 ROP Failure: 0x48A
Lid: 22630

The mailbox’s failed Move Request Log in the EMC showed that the mailbox had 2819 folders total, which seemed high.  There was also a Litigation Hold on the mailbox, which could be part of the problem.  (In the past, a Litigation Hold in combination with an Exchange bug had resulted in a 1TB mailbox on our servers: a 2GB mailbox with 1TB of redundant calendar data.)

Opening the mailbox using OWA simply to check out those 2819 folder revealed the problem, a massive number of nested Junk E-mail folders totaling about 250MB.

Junk E-mail

After the folders were deleted using Outlook and OWA, the mailbox moved successfully.  If deleting the folders using Outlook/OWA was impossible due to the sheer number of folders, then Outlook Cached Mode would have been tried next, then perhaps a server-side tool from Microsoft, and then possibly an IMAP client.

Posted in Uncategorized | Leave a comment

PowerShell error with Get-ADUser user -Properties *

After upgrading some of our servers to Server 2012 R2, we’ve discovered a bug in the PowerShell 4.0 Get-ADUser cmdlet. When running the command Get-ADUser username -Properties *, the cmdlet returns the following error:

Get-ADUser : One or more properties are invalid.
Parameter name: msDS-AssignedAuthNPolicy
At line:1 char:1
+ Get-ADUser username -Properties *
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (username:ADUser) [Get-ADUser], ArgumentException
+ FullyQualifiedErrorId : ActiveDirectoryCmdlet:System.ArgumentException,Microsoft.ActiveDirectory.Management.Comm
ands.GetADUser

Mike F Robbins researched the error in the following blog post, and determined that the issue occurs with PowerShell 4.0 run against Server 2008 R2 domain controllers. The issue is that two attributes, AuthenticationPolicy and AuthenticationPolicySilo, exist in a Server 2012 R2 Active Directory but do not exist in a Server 2008 R2 Active Directory. The Server 2012 R2 RSAT tools expect the attributes to exist in both environments, so an error is returned in the Server 2008 R2 Active Directory environment.

http://mikefrobbins.com/2013/11/07/windows-8-1-rsat-powershell-cmdlets-get-aduser-get-adcomputer-one-or-more-properties-are-invalid/

As Mike points out, a good workaround is to use PowerShell implicit remoting to process the commands on the domain controllers themselves. However, in the meantime I’m able to process the command from other downlevel OS machines, which is fine in my case for now.

I’ve submitted the case to Microsoft Premier who has confirmed the bug and escalated to the Platforms Team. I’ll post updates here as I get them.

Posted in Uncategorized | Tagged , | 3 Comments

HP Service Pack for ProLiant 2013.02.0 update broke NIC teams, VLANs using tagging

Networking was lost after updating HP’s Service Pack for ProLiant on two cluster nodes.  The servers were both BL460c G6 blades with two NICs that had first been teamed, then split back into two network connections using VLAN tagging.  (Multiple Networks had been assigned in Virtual Connect and then HP’s Network Configuration Utility (NCU) was used to team the NICs and configure the VLANs.)  Both servers were running Windows 2008 R2 SP1, with one being a file server cluster node and the other a member of an Exchange 2010 DAG.  The fact that the nodes were clustered is probably not important, but it was in the Failover Cluster Manager where the failures were most obvious, with our client-facing Cluster Network showing as “Failed,” while the connection used for backups remained functional.

On the first server, disabling/enabling the NICs in Windows’ Network Connections provided temporary relief, where the client-facing (Public) NIC would work for several seconds before failing again as seen in Failover Cluster Manager.  Disabling *both* NICs, then re-enabling the Public NIC first, then the Backup NIC second allowed both NICs to remain up.  This was not a permanent solution, obviously, so all networking as seen from Windows was torn down and recreated in Windows’ Network Connections and HP’s NCU, and the configuration was tested with a reboot just to be sure.

A call to HP revealed that this behavior was not unexpected.  Their advice when doing these updates is:

  • Break the team (and reconfigure one or more now-un-teamed NICs with production IP addresses if network connectivity during upgrade is required).
  • Perform the Service Pack for ProLiant upgrade.
  • Recreate the team.

The HP technician also noted that the previously-installed version of NCU was two years old and that there had been one or more significant updates in the interim.  When asked if frequent updates of the HP software would allow skipping breaking/recreating the team when doing these updates, he said HP’s advice would still be to do the above steps.

The second server proved more nettlesome, with HP’s Service Pack for ProLiant causing the production NIC to go offline soon after the start of the installation, losing the Remote Desktop session with the server.  The RDP session could not be re-established, and the server had to be accessed via iLO.  In iLO, the console session was not responsive to mouse or keyboard input, and the server had to be ungracefully reset (resetting iLO through its administrative web page did not help).  After logging in with cached credentials, the Service Pack for ProLiant installation was repeated “successfully” and the server rebooted.

After reboot, both NICs in Windows were showing as “unplugged” and HP’s NCU did not show any NICs at all, and both of these issues were after the NCU displaying an error that “The version of the miniport driver(s) for the following adapters are not compatible with the HP Network Configuration Utility software installed.”  Running the Service Pack for ProLiant a third time did not remedy the situation, and the CPxxxxxx.EXE file on the Service Pack for ProLiant DVD had to be tracked down and installed individually.  (The Service Pack for ProLiant was identifying the NIC driver as already upgraded, so it was not offering to upgrade it by default.  It may have been possible to force the upgrade from within the Service Pack for ProLiant, without tracking down the individual driver installation file on the DVD.)

After the reboot following the NIC driver upgrade, the NICs did not function normally on this server.  A tear-down and recreate of the team did not help as it had on the previous server either.  With eight NICs (FlexNICs) available to this blade, the solution was to create two separate teams (one for client traffic and the other for backup traffic).

With other blades in the chassis not having the luxury of eight NICs, the solution for those servers may be to remove teaming altogether.  Hopefully, this won’t be required and simply following HP’s advice to dissolve the team prior to the Service Pack for ProLiant upgrade and recreate it after will avoid the problem entirely.

Posted in Uncategorized | Leave a comment